I think every quilter struggles with this. I know I do. I seriously doubt any quilt I've made thus far has truly been square (and I don't mean this literally since I haven't made many square quilts, but you know what I mean).
If it were me I would put the borders on this quilt, trying to make up for the off measurments as best you can. Chalk it up to experience. It's more than likely you didn't have a consistent 1/4" seam all the way through.
I'm not one for redoing things so that's just what I would do. That advice may not help you though. :)