Muslin usually is a thin fabric with a low thread count. I use a medium cut away stabilizer, if the design is really dense I use a heavy weight cut away. You can cut really close to the design and still leave enough stabilizer to support the design. You might be hooping the fabric too tight and that would cause it to pull and pucker. Another thing to try is iron on stabilizer to prevent the fabric from stretching and distorting.