I use electrolytic rust removal all the time. All you need is a plastic tub, water, a little washing soda, a piece or two of scrap steel, and a battery charger. Works amazing, and doesn't harm anything else. It's how museums do it, and it's easy!
Electrolytic Rust Removal aka Magic
EDIT: This method actually doesn't remove rust, it converts it back into steel by removing the oxygen from the iron (and corroding the sacrificial metal instead). If you have bad, loose rust, it can just be wiped off easily after treatment.