If I understand your question correctly, I found I was having the same problem. What I observed was that it happened because the fabric was getting so close to the foot that I stopped holding it and thus it shifted. I solve this problem by doing two things:
1. Keeping a hold of the fabric in the back.
2. Use a tool (I can't remember the name of it - mine has two ends - on side has a point and the other a flat tip) to keep holding the fabric all the way up to the needle.
Joann