I don't know how you plan on using your jeep, but here is an answer. I hope it helps you.
The 2.5 is a very capable engine with just the right amount of torque for off road IF YOU KEEP THE VEHICLE STOCK. If you plan on going with larger tires and lifts, you will probably have to regear to 4.88 gears. As a daily driver, the four cylinder is fine but on the highway, it is terrible. Most jeepers with the four cylinder just have to get used to driving slower on the highway. Other than that, it is very similar to the six. Since you have enough experience with the six, I don't have to go into that.
Jeeps with 2.5 aren't in high demand... this is why you see them for sale everywhere. I would suggest driving one so you can see the power difference between the six and the four... then you can accurately decide whether this engine is for you.