You need to make sure your steering wheel is straight when your going straight. If your steering wheel is off that will happen. You need to adjust your drag link till your steering wheel is straight. Mine did the same thing till I adjusted it.
2003 Jeep TJ Silver 4.0L, Auto, ARB locked 5.13 G2 D44's on 35's
1998 Jeep TJ Sahara Black 4.0L 5-Speed, Replacing the frame
1997 "GI" Jeep TJ OD Green 2.5L, 5-Speed, lifted on 33's