Bent rim, bent axle...I've had both problems on different rigs I've driven.
Take your spare (you do have a same size spare, right?) and put it on the side they say is wobbly. Drive around a bit and get someone to follow you. No more wobble-it's your rim.
Take it to a tire shop with a 'spin balancer' and have them mount it up-and if the rim is bent you'll see it right off.