Exactly! Give yourself some slack!
I find that after I get a quilt finished, then I go back to where I think I should pick something out and re-quilt it, I can't find it. It looks worse at the moment than it will when the quilt is done. Sometimes washing the quilt will make things relax a bit, too.
I really love quilting with my rulers!