I teach sewing and keeping the fabric straight and guiding it is one of the things my students struggle with most. I do a couple of things to help them.
I have them practice on paper. I draw lines on the paper and they pretend the lines are the end of the fabric and they line their presser foot up to it and try to keep the foot on the line. Takes practice. And I have them use an old needle, nothing will dull a needle faster than paper, so you want to have a needle just for paper work. Remove it when you switch back to fabric and save it for work on paper.
We also practice controlling the speed at which you sew. If you have a machine with speed control, no problem, but if you don't, you need to practice controlling the speed with your foot. Slowing down can really help your accuracy.
We also practice guiding the fabric through. The fabric will go through with or without you, and probably reasonably straight through as well, you do not need to push it, pull it, or shove it through. What you need to do is guide it. Use the fingers of both hands. Your fingers should never be directly in front of the presser foot or to the left or right of the presser foot (I say this because you can line your fabric up to the left side OR right side of the presser foot, although most people line up to the right side). Your fingers should be 1-1/2 to 2 inches in front of the presser foot, guiding the fabric so it is staying in position. If it starts to stray a little, and you feel you are losing control, stop, be sure your NEEDLE IS IN THE FABRIC, lift the presser foot and move the fabric so it is lined up again. You may have to start and stop frequently at first, but you'll get the hang of it eventually.
And lastly, be sure your presser foot is at the right pressure. Some machines have a pressure setting for the feet. This is so you can accommodate different types of fabrics that may require a lighter pressure or heavier pressure of the foot for ease of sewing. If you have such a thing (your manual will tell you), be sure it is set on the proper setting (it's usually a dial and you set it on a number, for example, for "regular sewing", it would be #1). If you have a really old machine, the pressure may be able to be set manually, again, your manual will tell you.
Hope this is helpful!