I think that is what holds me back as well. I know Bonnie Hunter has an awesome scrap savers system and it seems to work well for a lot of people, but I think along the same lines as you do
What if I needed a 3" square and I cut it all into 2.5"?
LOL... I guess I wasn't much help, but wanted to let you know that you are not alone, and I think the best solution for us is to organize by color, not by size.