I would try quiltingshortimer's suggestion to try to shrink the outside edges. Hopefully it will work. Although this is not what you want to hear, If it doesn't shrink up, then you may have to remove the really loosely woven cotton. From past experience, I can tell you that it will probably shrink more than you think. I used a loosely woven muslin backing for one of my first quilts and it almost was a fiasco. The top got really puffy and wrinkled a lot more than I thought it would. One of my daughters loved it and I gave it to her. It almost became a dog bed.
I find that tightly woven fabric has almost no shrinkage and the looser the weave, the more stretch when working with it and more shrinking when it is laundered.
Edited to add: I wouldn't use a stitch in the ditch for the quilting, because this may accent the problem. Some type of overall design can hide flaws.