This way of doing SID has taught me not to fear or hate it anymore! As many have said: don't look at the needle, look at the foot!
You need a foot for straight stitch - one that has a narrow slot up to the needle hole: (I'm lucky to have a straight stitch plate, too.) You may notice some needle holes on one of the pictures - they were from something else.
This method will work for sewing on either side of the seam and allow you to switch sides while sewing if the "upper" side changes. Place the upper side just even with the slot in the foot:
And off you go! No need to go too slowly, in act that makes me "wibble". Practice a bit and Voila!: