I made a raggie with cheap flannel, too, but only used it as batting. The top and back are good cotton/muslin.
What I found was hard spots in the flannel that would make needle skip when adding quilting design to hold block together.
I don't like bare flannel. It feels creepy. No one in this house has ever had flannel sleepwear or flannel shirts.