Quick-Glo is a good polish and cleaner for metal. I used it to polish up needle and presser bars, chromed plates should clean up nicely. It removes rust better than the average polish, but steel wool with soap will get you there too and no need to go buy it if you happen to have some thing similar.
I don't think there's much advantage to soaking the spot in oil since a bit of scouring and hard rubbing is much speedier and needed to polish it up any way. There is stuff like evapo-rust but again, only if you already already have it. I'm just going on and on about the very basic stuff, but rust is this odd thing, very porous and loose compared to the intact steel and cast iron parts. Oil losens the rust and when joints, hinges and moving parts loosen up, it gradually flushes out. It can take days and weeks for rusty parts to run smoothly again, but they will. Your machine looks nice though, just in need of a good cleaning and a touch up of the finish here and there :- )