I also think that the 'permanent' effects of coloring with any type of marker depends in part on the fabric itself. How the thread is prepared prior to weaving may make it more or less accepting of markers, crayons, dye etc.
This theory goes out the window, however, for marking quilt lines... seems like sometimes these are permanent no matter what you do.