I just sold my hardtop, so I cant remember exactly what it should look like or I'd run outside and let you know, but from what you described it sounds like either you could put new weather stripping or rubber seal up front to make it more air tight when you tighten latches, or to check the latches and make sure they are solid on the hardtop. I dont think theres any difference from the 97-02 to the 03-06 version besides the air vents in the rear. Dunno of this helps as you might have already checked those but thats where I would look first if airs coming into cab from front
1998 Jeep Wrangler, 4.0 I6
33x12.50 Goodyear Wrangler Duratracs
3 inch Zone suspension lift, Rubicon Express adjustable front track bar, 1.25 inch body lift.