I get that sometimes also.You're shifting 'between' gears.In other words when you pull on the lever you go higher then needed to shift into 4hi.It will show 4wd on your gauges. Sometimes ,while it grinds ,even throwing it into park or neutral won't make it stop.Best thing to do is shut down the engine ,shift back to 2hi,or corect 4hi and restart.
Also,what i found to help when shifting from 2hi to 4 hi ,on the go,iso that you avoid grinding,i (and it's just a personal opinion) shift into neutral while rolling,shift to 4hi ,then shift (whithout slamin the gas
bach to drive.
As 4Jeepn said ,it can also be the shit rod underneath,but when adjusting that you have to be carefull not to screw up the 4lo.