I used a toothbrush and a little Dawn dish washing liquid in a small amount of water to scrub pencil marking lines from a quilt I made for a DGD. I had tried a few other things and this finally worked. Very tedious but it did work. I would use warm water with the Dawn if you try this. I know this isn't the same as your yellow chalk but just wanted to offer another option for you. I feel bad for you and hope you find a remedy that works for you.