Stitchnripper

Another suggestion is to use the blind hem stitch if you have that on your machine, along the sides. Less thread "bites" into the fabric and might lay smoother. I would test first if you decide on trying this.