I bought one of the mat cleaners, I don't know what they call them, it is round and has like sandpaper on the bottom. It takes off the fuzz from cutting and works really good. My mat doesn't have a smell, has your been wet or something?
I was curious so I took a clean white scrap and rubbed it on my mats and nothing came off. Do you really need to clean your mat?