If you have it cut with the weave, so that even though it appears angled, it's straight with the grain, I'd not worry about it.
I always go with grain as I rip when I'm doing backing. When I mount it on the frame, I use the rollers to help pull it into correct alignment. I'll pull and adjust as I roll the back on to the rollers, and then once it's all mounted and I can put a bit of tension on it, I'll roll it all back and forth a few times until it's laying nice.
Now, if you are saying that you have 120" on one side and 117 on the other - if you only removed selvages to seam it, I don't see how that could even happen.