I belong to a few groups that do gift exchanges, but they make it clear that they are voluntary. If you don't wish to participate, you don't. I usually don't, but I enjoy seeing other people who do enjoy that activity.
One of the neighborhood ladies has regular bunco parties. I don't care for bunco, but I do enjoy the company of the other ladies who come to the parties, so I always try to attend. Some of the prizes are little trinkets/knick knacks that the hostess picks up here and there (maybe through gift exchanges with other groups???). When I win one I have to pretend to be pleased, but in reality I'm trying to figure out what to do with the object when I get home. Most go to Goodwill, or into the trash can. I don't need more junk! That's the main reason I don't usually participate in gift exchanges.