Originally Posted by
Tartan
Rather then putting any moisture on the rusty pins that could make the rust stains worse, I think I would use my wire cutters to cut up the safety pins and carefully remove the pieces.
And if the pieces are still belligerent, try grasping with a needle nose pliers or hemostat to pull the bugger out!