Someone (not sure who, it was a quilting teacher) said to use the mat to line up fabric so it is straight, but to use the ruler to cut and alway leave the line on the ruler on the fabric.
I think as others have said, as long as you are consistent, and your piecing is accurate and your blocks the size they are supposed to be, it should all be fine