I had a student who had the same problem. The lengthwise grain of the fabric doesn't stretch, but the width does. That's why it was OK sewing in one direction, but not the other. If you had a walking foot, it would sure make a big difference. It's like putting feed dogs on top of your fabric. Using a textile spray adhesive is also very helpful.
Good Luck to you!