I'm interested in responses to this question as well.
For some you can tell, e.g. it might be slightly "grittier" or smoother than the other side, but then there are so fabrics I can't tell. I figured that if I couldn't work it out, then maybe no one else would notice either :)