Also, don't have a death grip on the ruler. You need to hold the ruler in place, but it still needs to be loose enough that you can still move your fabric. Same thing with the ruler pressure against the foot. Needs to be able to glide.
I'm assuming that you can FMQ in all directions without the ruler?