I've used the Crayola washables with success, as well as the blue and purple "quilting" markers (note that the purple disappears after only a few hours - or even sooner if the humidity is high). I've also used the chalk, but as someone else mentioned, the white is the better one to use - the blue chalk is sometimes difficult to remove.
And I have even used a regular #2 pencil, that has been sharpened to a very fine point. I use this if I'm going to use contrasting thread - the darker thread usually hides any pencil marks that don't wash out, but I've only had that problem once (I ironed it - won't do that again!!)