This happend to me as well. I would move forward with the rest of the blocks, just making sure that you stay as close to your 1/4" seam as possible. Then, after you see how you have to set the blocks then you can determine where you have to make a compensation. For example: You could trim all the blocks to the same size. Or depending on your sashing (if you have one) you may be able to compensate there. You might also try blocking some of the blocks. Sometimes that can help as well. But I would not do anything until all the blocks are done and you see how they are to go together in the quilt. That will help you to determine how to correct the blocks that are off.
Is the pattern online anywhere so that we can see what you are working on?