I square up each block before sewing them together. I used to try to square up the whole thing but, like you, found it hard to do, and I decided it wasn't worth it. I trim off the extra backing and batting as straight as I can, and then sew on the binding.
I've also noticed on smaller things, like on some cat pads I've made that get washed often, that they shrink unevenly. I prewash the fabric, but still after many washings they'll frequently end up looking uneven. I'll walk by them and think to myself, "Good thing I spent all that time getting the edges straight and the corners square!" I don't wash my quilts very often and haven't seen them get off, but seeing the pads warp has made me worry about it less, because I figure over time they're going to become off a bit anyway.