Did you use quilting flannel or regular PJ flannel?
I have made quite a few rag quilts, but all with quilting flannel, all were washed before being given away.. I also would never wash them with towels, too much lint in the towels.
Actually the above is not quite true. I made several dog quilts with a UFO I picked up in a thrift shop. The flannel was not quilting flannel, and it pilled, bled and did not look too nice, but I figured dogs do not care too much about how their quilt looks.
It is too bad the fabric did not stand up as expected.