Yep, all you are doing by holding the throttle is getting it past the idle circuit. Ain't like a carb where pumping the pedal adds more gas.
Pull the IAC, clean it real good, and also spray carb cleaner into the port the IAC goes into. Then open the butterfly on the throttle body, look for a deposit ring around the bore, below the butterfly. More carb cleaner, and a toothbrush if it's really stubborn.
Watch when you reinstall the IAC. If you bugger up the O-ring seal it can bind the IAC pintle valve and you'll have all kinds of idle problems.
97 Sahara, 74 Renegade, 72 CJ5, 55 Willy's (the Beast) Boss 302, 6:34 gearing, Bose 901 speakers.
God I miss that Jeep. Forgot the kinda Jeep, 80 SX4.