Your first entry told what you should have done different.
HEAT SET THE BLOCK WITH THE CRAYON COLORED SIDE UP
WITH A PAPER TOWEL COVERING IT.
When you put the crayon side down you transfer the color to whatever is under it. (you said "face down" so I'm thinking you put the colored side down, is that right?)
Bill's BonBon mentioned sandpaper, you can color directly on the sandpaper then lay it on your fabric and press. The color transfers from sandpaper to the fabric. You can also color on regular paper and transfer it to the fabric by laying the colored side of the paper face down and pressing.
If you do sandpaper or paper transfer, remember the picture will go from the paper to the fabric in a mirror image. If you have writing on it the writing will transfer backward. Just another way to do the crayon on fabric.