Have you tried Mr. Clean magic eraser? I have used it on pencil marks on white fabric and it works like a charm. Try on a test swatch.
You could also try oxyclean liquid mixed with water in a spray bottle. Spray onto fabric and rub with magic eraser. Definitely use a test swatch.