The direction of grain matters a lot more in making clothes than in making quilts. Once the whole quilt is finished I can't see that it matters at all (unlike clothes, when it matters the most). While quilting, maybe it is less likely to bunch up if you're traveling in the direction of the least stretch, but since in general people are quilting in multiple directions on the fabric, it's clearly possible to get good results regardless of which way you're going on the grain. Starching the fabric would probably make more of a difference. Heck, I don't even starch and here I am recommending starch.
I know you're already made your decision, but I'm just "thirding" what's already been said.