Were it me, I'd disassemble it, soak it in Evapo-Rust for a bit, then use the fine wire brush on my Dremel tool or 4/O steel wool on the rusty spots.
That would clean it under the tension spring as well. You never know what kind of junk is under the spring.
A good oiling then a wipe down, reassemble it and it should be ready to adjust the tension.
Glad you got a bobbin case for such a reasonable price.
Joe