Oh, dear. This happened to me as well, and I couldn't get the marks out. They did fade over time, however.
Quilting markers are really, really individual. There's no marking method that suits everyone. Some of it seems to be about body chemistry. I think that when pencil works for someone, they are able to mark very light lines (which is partly a dexterity issue) and also able to see very light lines (which is about eyesight, lighting, and fabric colour). Frixion is one which is notorious for working fine for some people, and with other people the marks come back later. I can't use chalk, it brushes off way too fast with my quilting methods. Personally, I like washable marker pens, the sort aimed at children. But not everyone does! My point is to always test, and don't assume that what works for one quilter will necessarily work for you.