Having the same problem with my 98 TJ 4-cyl. Found this useful information below, while searching the web.
Heater Core Flushing Made E-Z
I had a problem with my XJ's heater not providing very much heat for a long time,
and it continued to get worse as time went on, until the heater barely got warm at all.
I had flushed my cooling system (annually) but this did nothing for the heat problem. The design of the XJ's closed cooling system, doesn't allow the routine engine flushing process to really get the crud out of the heater core, so it tends to build up over the years and eventually reduces the flow to near nothing.
Since I had just flushed the entire system, and I didn't want to drain and refill the system again, So I figured out a way to flush just the heater, and leave the rest of the cooling system intact. I went to the hardware store and bought 2 feet of 5/8'' and 2 feet of 3/4' heater hose, and a 5/8 & 3/4" female garden hose coupling replacement kits (look for the type that use hose clamps), and 2 rubber stoppers (or corks) to fit snugly inside the 5/8 & 3/4' hose.
The only preparation that needs to be done, is to install the female hose couplings on their respective pieces of heater hose, and park your Jeep where the flushing won't make a mess (preferably not on the grass, antifreeze is a great de-foliant and will kill your lawn). The 2 foot lengths of hose, gives you enough to route the flushing output down out of the bottom of the engine compartment, so it won't get the gunk all over everything.
Now remove the vehicle's heater hoses from the core, one at a time, and quickly cork the end of hoses before any (much) coolant leaks out. Then attach the two short pieces of hose to the heater core input/output, using the clamps from the heater hoses to secure them in place. Now starting with the lower 5/8" hose, hook up the garden hose and give it a full blast shot for about 30 seconds. This should unseat gobs of crud. Next, go to the upper 3/4" hose for a shot, then back & forth a few times 'till the water runs clear, ending with the lower hose.
Now you can turn both hoses up, and fill the heater core with water, then starting with the lower hose, pull the flushing hoses off, and hook up the heater hoses quickly to avoid any (much) loss of coolant. Just top off the reservoir and you're done. (The amount of water in the heater core will not significantly change the ratio of coolant. If you're really concerned about that (or live in Alaska), just use antifreeze to fill the core and the top off the coolant bottle.)
The whole process took me less that 20 minutes, (not counting the trip to the hardware store...), and now I have H-E-A-T! So much heat in fact, I have to turn the temp control down to 1/2, even when it's below freezing outside.
Have fun, and keep warm.