Not only can coloured chalk stain your fabric but it can be observed into cotton quilting thread. The blue chalk I used on a quilt many years ago is gone from the fabric with the repeated washing but the white quilting thread still has a blue tinge. Try different cleaning solutions on scrap fabric to see if you can get it to fade enough that you can live with it.