If I have fluffy, poly scraps, they are kept for stuffing. I keep a few small pieces as cleaning cloths, I use larger pieces for potholders and small projects.
One of my favorite things to do is to take 12-20" squares of batting and use them as foundations for scrappy blocks. When I put the tops together, I use narrower pieces to go behind the sashing and border strips.
I do sew pieces together to make larger ones.
However, both time and space are limited, so I think it is important to reach an understanding with yourself as to what is worth keeping....then follow your own rule.