Instead of moving the alternator, look at whether the pulley is all the way back. Maybe it needs a shallower spacer.

To move the alternator
* Use a washer behind the upper (adjusting) bracket
* Slot the bolt holes in the lower bracket, but you might not be able to if it is cast iron. In that case hacksaw or grind a bit off the front off the bracket (where the alternator attaches) and use a washer to take up the slack on the back.