It would help to know what sewing machine you are using. Is it possible that your fabric is not covering the feed dog on the right side? This can happen when trying to sew a quarter inch seam with the needle in the center needle position with some of the modern machines that are capable of extra wide stitches. It also happens trying to sew a quarter inch seam with some vintage machines that are left homing when sewing a straight stitch.
Rob