Start at the other end, the engine. It sounds like you have a vacuum leak.
If it's a 4.0 look at the intake manifold, towards the rear. You'll see a vacuum hose coming off of it to the firewall. Follow it. It splits off about mid cowl. One half goes to run your vacuum controls for the doors in the heater-a/c case. The other half runs over to and underneath the battery tray, to the vacuum reservoir. Common problem.
You start up, lots of vacuum pressure, enough to fill the reservoir and compensate for the leak. Start down the road, vacuum pressure drops. Now it can't cover for the leak. Once the reservoir runs out the leak is more then the engine vacuum can supply, and one or more of the doors in the case close and you get no airflow through the vents.
If you get a bad enough leak, only air flow will come out the defrosters.
97 Sahara, 74 Renegade, 72 CJ5, 55 Willy's (the Beast) Boss 302, 6:34 gearing, Bose 901 speakers.
God I miss that Jeep. Forgot the kinda Jeep, 80 SX4.