If it works for me, I'm happy. If it bothers me, I do something about it.
I have exactly the same philosophy. If it bothers me, I fix it. I tend to be a pretty careful and accurate piecer so I don't lose many points, etc. But I do lose some and seldom go back unless I REALLY missed the mark, or if it is part of a larger design where it will be noticeable. I want to enjoy the process, not create one where I feel like I am never measuring up.
I do enough of that to myself in my professional life.