The main thing I have learned after more than 40 years of quilting is that fabric is not a "stable" medium. It stretches, shrinks and is often not straight to begin with. I tear all of my fabric and also prewash it and it is still not straight. In the manufacturing process they wrap it on the bolt crooked and it seems to stay that way. The only thing you can do is "press" it firmly and starch it if necessary and then take your chances. It does help to support the item you are quilting as much as possible. That is one reason I am not interested in machine quilting something any bigger than lap size. Physically dragging it around under my machine is more than I can handle because of health issues. If I have to work on anything bigger I surround my sewing setup with tables that will support the quilt or whatever I am working on.
If it helps to make you feel better, my husband is a macninist who spends his working hours cutting up metal. It also stretchs and shrinks from changes in tempature. He actually carries some items from one end of the shop to the other in a cooler with dry ice to maintain their size until he can cut them accurately. If metal isn't stable how could fabric be?
Finish your project and be happy with the color and design you have accomplished. No one is perfect.