You sound like me, except that I only use the 1/4" foot without the fence. That works for me. I still veer off at the end of seams a bit too frequently, but I resew the skinny part of the seam to get the 1/4". No one sees it when it's quilted, so why not?
As you've probably guessed, I am not quilting "heirloom" quilts!