It can be a problem. I have my computer set up with three monitors. There are definitely differences in the colors, brightness I'm seeing. Sometimes I purposely switch a picture from one of the more "dull" monitors to the one where the colors show up brighter.
So, your problem can be a big one. A person must have at least a small sense of adventure and be willing to take some risks in order to buy on line...really anything, but fabric is one that is a definite risk. That being said, I'm usually quite happy with what I've ordered.