One little thing that can throw your stitch off pretty badly is not having the needle pushed all the way up.

If you use Singer needles, they're a little smaller at the top, and can wiggle their way out.