It probably is just the stitching itself. If the tension is not correct, the fabric will draw together a bit on a long seam. Another thing that may be happening is that the fabric is shifting. Did you pin it a bunch? A walking foot may also help.
When I made drapery panels, I didn't sew the hem to the fabric, the panels were lined and I sewed the home dec fabric to the lining by hand.