I use 3 layers of flannel, all cut to the same size, in my rag quilts. I do wash everything for many reasons - shrinkage, bleeding, and chemicals in the fabrics. With flannel I also find out whether the fabric will pill, and if it does I use it only in the middle layer. Washing does not affect the ragging at all. I can't really think of any reason not to wash the flannel, unless it's already cut into pieces. However I don't really think shrinkage of the flannel will be an issue for you if you choose not to wash.