I have done a few quilts with panels in the center, and sometimes the sides do not make it possible to add blocks. Same as ckcowel above, I would add borders as needed to make it easier to determine the number of blocks required to go around the panel. I have added borders on opposite sides to make the design look even. This type of border is called "spacer border."