Yes, you can remove the spare. I wouldn't bother removing the carrier as a stock one only weighs a few pounds.
What size were your original tires? If it's close, I'd keep the spare until you can afford a new one as if it's only off by an inch or so, it won't matter (UNLESS you run with lockers and use them with the spare on, DON'T DO THAT, as it will destroy your locker).
I always put 35 PSI in my spare. Two reasons. First, it will leak a small amount of air over time and that way I shouldn't have a flat spare when I need it. Second, I can always let air out when I mount it if I need to (I run 26 lbs in my 35" tires). No weight on the tire and inflated to it's advertised maximum air pressure is what to aim at, just remember to let some air out if you need to.