To prevent this type of problem, I heavily starch the backing before layering. I use a 1:1 solution of Sta-Flo liquid laundry starch and water, "paint" the solution on using a large wall painting brush, wait a couple of minutes to allow the starch solution to be absorbed, toss in dryer, then iron with steam. This stabilizes the backing fabric enough so it will not fold over itself while machine quilting.
Another thing that helps a lot is spray basting because it "glues" the layers together at all points -- not just every 4 inches where there is a pin, or every couple of inches where there is a thread basting stitch.