I have mixed washed and non-washed fabrics in the same quilt. I haven't had any problems. But I'm sure there are some people who have had problems. If you're buying pre-cuts, or fat quarters, you can't really wash them. The most that I would expect is for some wrinkling at the quilt stitching lines. But that would just add to the charm of the quilt.
There would be more of a problem with cheap flannel. I would probably wash that before quilting.