You can try some bar's stop leak if it's not too bad. Make sure you follow the directions as I think it requires you to use water only to let it settle properly.
Denis A. Baldwin
1991 YJ 99 Vortec 5.7L, SM465, NP205, 14 Bolt Rear, Dana 60 Front, 5.13 Gears, Detroit Lockers, 395/85R20 46" Michelin XML tires on 20" XD Wheels, Cherokee Offset Springs (10" front to back stretch total), Bilstein 5150s, 1in. BL, custom cage, Warn M8000, Double D Dash/OHC, 4Low Front Stinger Bumper, DirtWorx Rear Bumper WTC, Tube Fenders, Comp Cut, 1 Ton HD Kingpin High Steering and PAID FOR.