I resorted once to using an old (not current) tooth brush head in my electric toothbrush. I used it lightly, hoping it would brush away the chalk, because I was desperate. It worked reasonably well, but I think would depend on colors. If you think you might try this idea, I would recommend you use it on a piece of scrap fabric first, just to see if it would work, and to make sure it wouldn't affect the fabric surface.