Flannel is cuddly, but I always go for longevity with woven cotton. I just don't believe flannel will hold up as well, over time and many washings.
One thing I LOVE for backing is a good vintage sheet, still in great condition. Estate sales usually have sheets that came out of the person's linen closet, often unused! Good sheets have higher, denser thread counts and feel so amazing on your skin.