How were you planning to remove the chalk with your original quilting plan??
Just follow the same plan.
I find that a lint brush for clothing does a pretty good job.
The type with the fabric with rough edges that catches the lint ... brush it across and pretty soon the chalk has disappeared.
Like anything ... test it out before you do it and regret.