I have never had that kind of experience! I usually get sympathetic encouragement and praise for what I do, and even offers of fabric or help!
I also am careful to keep my charity fabric separate from my personal stash, but I have an agreement with the head of the group for whom I sew that I can swap out equal quantitites when I find something more appropriate in my stash than has been donated to the charity!