I figure if the mark comes back in the freezer, it means *something* is still on the fabric. Whether or not you can see it, it's there. And if it's there, it could degrade the fabric or someday, come back even without the freezer.
Some say the wash takes it out and that's fine but for me, I prefer to use something like chalk that I know comes out 100%.