I have done it a bit, but that is because the only thing I don't prewash is precut strips. My Gypsy Wife quilt had quite a few unwashed strips, and some of them were used WOF. Didn't notice any problems once it was quilted and washed.
I use poly batting which does not shrink up like cotton batting, so I think any problems would have been more noticeable. If your batting is going to shrink up, you aren't going to notice much with the fabric as it will all end up crinkly looking.
I don't worry too much about shrinkage, and have no problem using washed/unwashed squares together. I prewash because I've had problems with dies bleeding.
Mine are already packed up - so I'll just tell everyone that all of mine are prewashed.