I think you could remove the pins at that point; the layers would be unlikely to shift with the SID in 6-inch blocks.
If you do have any puckering when you make the Xs, I would try spray starchig the top and backing a couple of times before continuing. (No need to iron.) That should stabilize the fabric and prevent puckers from forming