Whoops, it doesn't matter what side the resistor is on, just that it's there on each LED. The problem is the current running through your LED's. 470 ohm isn't enough. V=IR. V=volts, I=current in amps, R=resistance in ohms. Your alternator while running typically outputs 13.5 volts but can peak at 15V or something very close to that. Now the rated maximum current that goes through the LED's you have is unknown but the forward amperage is 20mA, or 0.02 amps. At typical system voltage of 13.5 and full bright on the dimmer pot, with 470 ohm resistors the current passing through your LED's is 29mA. At 15V it's 32mA and at 12V they are getting 26mA. Using the dimmer, your LED's will start to turn completely off at 9.4V. So until we know the maximum current that the LED's can handle, we can't say for sure what resistors to use.
It looks like the dimmer circuit is not PWM and just doesn't play nice with LED's. Did you notice any color shift when the brightness changes or difference in color between one LED and the next? Could just be cheap LED's like I said earlier or your connections are going bad from vibration or a weak solder joint.