I've been told that it is necessary to 'straighten' the cut edge of the fabric you are rotary cutting after 3 strips have been cut. So, unfortunately, we all have the 'remaining fabric not straight after cutting' problem.
What has produced the best results for me is spraying the back of the ruler with spray baste, then rub a lot of it off. It must be a bit tacky when you're done rubbing it off the back of the ruler. The spray baste makes the ruler stick to the fabric better, so there is much less shifting. The spray baste glue wears off, over time, but when it does, just repeat the process.
Another thing that I do that really helps me cut straighter strips is to press a sharp crease at the WOF fold. It reduces 'elbowing' significantly!