I don't use any softeners on my fabrics when pre-washing. Partially because most of my fabric goes into the stash and I never know when I'm going to want to do something like adhere freezer paper to it or paint it. I definitely don't want to have to re-wash anything I've pre-washed!
I also wanted to mention - flannel, I pre-wash TWICE, if not three times. It can shrink so much and loose so much "fuzz", and sometimes flannel isn't that great quality but I can't really tell until the second or third washing. If the flannel still looks and feels good after 2-3 pre-washes, I'm confident it's the good stuff.
I'm usually using it for quilt backing, though, where there will be a big expanse of it and any pilling or loss of nap will be really noticeable. In a rag quilt, I think you can get away with a bit more.