My thoughts (worth thousands of dollars but free right here for a limited time):
A) the world of hipo cars has many AWD offerings. The reason is to increase traction on slippery surfaces. These cars run AWD at all times, with variable amounts of power being sent to front wheels on an as-needed basis, but always some power going there. If there were no benefit at speeds higher than 40 mph, surely they'd figure out how to decouple the front differential and improve mileage, given the lengths they are going to now to improve mileage. And yes, those cars are mostly pavement only, but slippery pavement and slippery gravel or dirt are both traction-compromised surfaces.
2) Obviously, with our 4WD setup be very certain you're not encountering any intermittent high-traction surfaces like stretches of solid rock while in 4WD
3) Use it or lose it. I haven't heard this very often of Wranglers, but when I was shopping for my first Jeep, an XJ, in the late 90s, word on the street was "Be sure and test it in 4 Hi and 4 Lo because a lot of people buy these (in Atlanta) because they're cool looking, but don't ever shift into 4WD. Lube never circulates and the front diffs become locked up." I never found one like that, but it makes sense that the front diff is made to be used occasionally and it's probably worse for it to not get used very often. But that could just be an Old Jeepers' Tale.
4) Most importantly, c'mon, admit it; it feels manly as HELL to reach down there and wrestle that lever back into 4Hi, exceeded only by how great it feels to pull it on back into 4Lo. And once you shift it the Jeep feels different. It drives with a new restlessness, like a dog who's felt you hook the leash onto its collar and knows a Big Adventure awaits, or a horse that's been saddled and is eagerly pacing around in circles waiting to get out into the Big Wide Open. There's a more tactile feel to the steering as more inputs are coming up through the steering column and into your hands. There's a slight resistance to turns that you don't experience with 2WD, as though the Jeep itself has ideas of where you should both go and strains a little at the leash to let you know about it. There are new sounds coming back at you from a differential that, no matter how often you get to the trails still only gets used a tiny percentage of the time its brother out back is allowed to run.
All in all, why the heck wouldn't you want to drive in 4Hi at speed on a dirt road? I'm literally ALWAYS looking for a reason to do so.