If it's dirt, I've successfully removed spots by vacuuming them with a piece of old hosiery over the vacuum hose & then used a Tide Stick & a bit of water with a white washcloth. You just have to be super gentle. Tartan's idea of a Q-Tip is a good one. CLR will get rust out of fabric, but I don't know that I'd want to keep handling the fabric for so long after I treated it with chemicals. Perhaps you could soak it after the piecing is done if the marks are on the back of your quilt top.
Mostly I would leave it until you finish quilting it. And if you can't afford new clips, cut little snippets of fabric to wedge between your quilt top & the clips.