If you fold the binding over and then use clips (some use hair clips) to hold the binding in place you will probably not need the fray check. The edge is fraying because it is not protected. How did you first sew the binding to the quilt? Most sew the binding on with their sewing machine, then fold over the binding and then hand sew the second side to the quilt. Just wondering what process you have used so far?
Pam M