I'm not sure I follow your description but could it be that half the panels must be reversed. The bottom and top halfs are mirrored imaged....could this be the problem.
Are you winging this or following a pattern. If not following a pattern, try to find some instruction and see if it wil give you a clue.