I use a med grit sand paper. I just cut them into strips and glue on one or two strips off to the side with an Elmer's glue stick. It lasts a long time.
A quick fix you can also use is to place emery boards under the ruler. they will create a friction and stay put until you move the ruler. I use this all the time. You could glue them on if you are using large rulers.