Stock will do you just fine on reasonable terrain, as long as you use your brain - don't set yourself up for failure or parts breakage needlessly. Stock will not do fine at hardcore off-road parks or other such silliness, because that silliness is intentionally designed for highly modified vehicles and idiots who like to break things on their inadequately modified vehicles.
Carry a proper recovery kit, with a good shovel (or two, if you have a passenger). Know (or learn) how to fix basic stuff. Then go have fun.
If you're looking for your best bang for your buck in terms of upgrades, a good set of tires will make more of a difference than anything else at this point in time. A YJ with 30-9.50-15 tires, a bit of additional underbelly clearance (think: high clearance trans crossmember/skidplate) and a lifted gas tank would actually be quite capable. The low-hanging trans crossmember is going to be your big limiting factor right now.