My sister collected salt and pepper shakers for years and everyone knew it. She was collecting specific ones but we didn't know the difference. We were just happy that we knew what to get her for holidays and suveniers from travel. After about 15 years of getting s&p shakers she didn't want she finally started telling people-no more, please.
She told me that she had only started collected them because she saw a couple really cute, old ones at her grandmother-in-laws house and she gave them to her. It really wasn't supposed to be the start of a great collection.
So tell all your button suppliers that you appreciate them thinking of you but you don't really do anything with buttons. Just be honest and up front. They may be embarrassed that they just assumed or they may be sad they no longer have a simple gift to give you.