Well, if it were me, and I had a headache every month for something that's not only not necessary, but is supposed to be fun, I think I'd back out. Maybe just go to the get-together but bow out of the gift giving. or just make a dessert for the gathering and that is your swap gift.