Sure, but once the tub mounted carrier is in place you won't even know the holes were ever drilled. Besides, bumper/swingout combos weigh a ton and are prone to rattles unless they have an excellent latching mechanism.
Rubicon Express 3.5" SF Kit / American Racing Baja's / 33x12.50 BFG MT KM2's
It has a very solid latching system, no rattle at all. It also gives you two different options as far as mounting the spare. You can mount it in the "inboard" postion, to keep it closer to the body, or mount it in the "outboard" position, to leave room for 2 gerry cans and a Hi-Lift. It holds my 33" just fine.
www.Jeeperman.com has the best I've seen. The RockHard4x4 is basically the same....they ran with the Jeeperman design. You can't loose with either, but I would personally support Jeeperman.
The Jeeperman ships with frame tie-ins included.
Also take a look at www.shrockworks.com. They have a great rear bumper/tire carrier. The LOD would be a great option also.