I'm not a scrappy quilter by any means. I also only buy for projects as I just don't have room to store anything. Beyond that I know if I bought a piece of fabric just because I liked it, I'd never figure out what to do with it.
Fortunately for me, there is another member here on QB that I have become friends with and she is a very creative quilter and sewist. She does lots of charity quilting as well as other types of sewing. When I finish up with a project (or 2 or 3), I'll gather up all my leftovers and give them to her - including batting scraps.
I would not feel guilty in the least tossing this but I am happy someone else is able to use these things and get enjoyment from them.
Win-win for both of us.