I would probably use Retro Clean. Here is the website for it:
http://www.retroclean.com/q&a.html
Have not used it myself, but haven't seen any negatives about it.
If it's really mildew, I'm not sure anything will get it totally out. If that's the case, you might be able to retouch the spots with tiny dots of fabric paint in a color that matches the background. Art supply stores carry that kind of paint.