I suppose I have never really paid attention if my fabric moves or not.... I'm like sewingpup, I just toss my fabric on the mat and however it lands, that's where it stays. (okay, maybe i'm not quite that carefree, but you get the idea!). if my fabric moves when I place the ruler, it doesn't really matter. I just shift my ruler around until its lined up. perhaps my fabric moves around like yours too, but it has never registered on my radar.