I haphazardly prewash so most of what I make has a mixture of some prewashed some not. I use Warm and Natural/White in all the things I make which also has a scrim that is not cotton. I wash everything after it is done in warm water and detergent to get rid of the glue I used in basting and dry it in the dryer. I throw in a couple of color catchers for good measure but rarely do they come out with any significant amount of color on them and shrinkage is not a problem I don't think not prewashing is the answer to your problem. My guess would be either the batting but more likely the muslin.