Brownies? you said brownies? You have my attention!
The discrepancy comes when you stitch, and fold the seam back.
A bit of bulk from the thread and a bit of bulk from the fabric (each layer in the seam)
and a bit of loss in the 1/4" seam and it's soon out of whack.
You did the right thing by shifting your needle over, to make up for the adjustment.
One way to help figure it out is to take three strips 1-1/2" wide and stitch.
Press the seams and measure.
If all is good, then that should equal 4-1/2", right?
Now re-do it with three more strips, but adjusting your needle.
Press and measure.
Keep re-doing until you are where you want to be.
In theory ... this strip stitch and measure technique would be better done with more than three strips, as if you are just out a hair on each seam, then it would not equal what it should be.
Of course ... this all gets thrown out of whack if you work with a different fabric and/or thread.
One of the reasons that I really like Aurifil thread ... thinner and doesn't mess things up as much.
Make sense?
.............. now, do I get my brownies?
(now let's see, I could be there in what do you think probably about 2 hours?)