I used both a plastic scrubby and on the really embedded stuff I used one of those green scoring pads (I get them at the dollar store).
BUT, a better solution is to designate a mat specifically for cutting batting and fleece as well. So if you have an older mat, kind of worn, make that the one you use and get yourself a nice new mat when they go on sale at Joanns. That's what I finally ended up doing.