For the parts you removed you could use alcohol. But alcohol dissolves the paint so for the parts that are inside you could use liquid wrench. It's mostly kerosene and very stinky. I would try sewing machine oil first. Squirt some on and then rub with a rag, let stand, repeat, etc. Requires more patience but I really hate the smell of liquid wrench.
Not to minimize your complaint, but your machine doesn't look so bad! The gunk looks like dried up oil. There doesn't seem to be rust, and that's the worst.