I'm sure I have about 20-25. The 'rule' is to use them.
In 2009, I decided that any ruler not used by the end of the year was going. I stuck a piece of tape on the end of each ruler as it was used. At the end of the year, 8 non-taped rulers left to new homes. In a couple of cases, they were duplicated rulers that I would rarely use in the first place.
Now, this is different from templates. I have a boat load of those as well and they are actually harder to part with. For instance, I don't need my Thousand Pyramids or Apple Core templates after this year, so they will go in the box for swap.