Honestly I've never had a problem using multiple rulers OR multiple machines on a project, and I do both frequently. Not sure if I'm lucky, extra careful, or just so inexact in everything that I just don't notice, but it's never been a problem for me. I don't understand why it would be, as long as your rulers aren't made wrong (and haven't been shaved down from wear) and you take the time to check your seam allowance before you get going.
I suppose if you have a really worn out machine your settings might somehow "wander", but that's a separate issue. My travel machine is always kept set up for a quarter-inch seam, but I check it periodically anyway - I think that's just a good habit to be in.
I've checked my rulers in the past and they all agree on what an inch is, and so do the lines on my big mat. Maybe some mats expand and shrink with temperature changes? I think I'd be inclined to return my mat if the markings weren't accurate. I haven't checked my smaller travel mat, but I almost never use the lines on that one so it kinda doesn't matter. I'm curious now, though.