A couple of years ago I made a Wizard of Oz with a panel in the center. As I built the rows around the center I realized that the center wasn't laying straight. It was puffy. I should have stopped at that point and ironed some fusible interfacing on the back to keep the panel from stretching but I didn't. I continued to finish the quilt and the center is still puffy. If I were you, I would use fusible interfacing on the back of the panel.