It might be your backing. Sheets are woven different than quilting cotton. I believe I've read that sheets have a higher thread count so they are much tighter than quilting cottons.
I've hand quilted with 100% poly and it was ok. I've hand quilted with wool and it was like butter-and looked beautiful.