From the looks of my sewing room, I must never throw things away. I just keep moving everything from one place to another when in straighten up the room. I've tried taking somethings to our local charity resale shop for craft goods, but I take in one box and come out with two.
I think when you try to use up your scraps they just expand when exposed to light as you dig through the containers.