It depends on the batting and the quilting. Once the three layers are quilted sufficiently, batting controls shrinkage. Years ago I took a class with Harriet Hargrave. She had made a quilt out of unwashed flannel to prove to us that fabric shrinkage is really not a concern when you are doing moderate machine quilting. She measured the quilt before and after washing, and it shrank only the usual amount of the batting (Hobbs 80/20).
Where you can run into problems is if the unwashed flannel quilt is tied. Also, if quilting lines are far apart, it's possible for the flannel between the quilting lines to shrink enough to distort the quilt. Even then, though, it's usually possible to re-wet the quilt and block it, then add additional quilting lines.