GiGiGreen ... First of all, don't dismay, as many of us have been down the same road as you are on now. You sound like you have a good sense and willingness, by suggesting ripping things apart, if need be! Sometimes too, we need to be a little forgiving of ourselves, and decide whether it is really all that bad? Or are we building too high of expectations, for the level/skill we are at? Only you can decide on that!
Without a little more detail ... better still photos ... it's hard to know exactly what your problem is.
I'm guessing that it's that all blocks are not the same size. Yes? No?
Consider this ... if with each seam you are just a hair wider then all those seams together can add to different block sizes. Worse if out two or three hairs, or more! IOW the challenge is, making that perfect 1/4" seam every time! As GingerK suggested, if the difference is minimal, you can put the two together, and solve the problem as she suggested, along with a good pressing.
But if it is not minimal ... it's back to achieving that perfect 1/4". One way is to measure after every seam.
Sounds horrible ... but the first log cabin quilt I made was in a Quilt In A Day class. We were using one of Eleanor Burns' early Quilt In A Day books, where she tackles that issue where after every seam you press and measure to make sure your block is on size. If not, correct it! It really wasn't all that tough, once I got onto what we were doing. Plus, I soon got a better handle on 1/4" seams and felt rewarded every time that my log cabin measured up!
Otherwise, waiting until the very end, with a fully assembled log cabin, you are in trouble.
Show/Tell us more, and you'll be sure to get lots of help!
Good Luck ... and enjoy your quilting journey!