i always hold my fabric along the left of the foot as it goes under the needle. sometimes the feed dog will pull your fabric one way or the other and it just needs some guidance. some use a pointed stilleto to help the fabric go straight.
perhaps a scant 1/4 inch seam would work for you better. that is to stitch just on the inside of the seam line.