I understand your dilemma about possibly needing one of your flannel cuts in the future. If there's any question, why not keep it? It's okay to want to keep it. Just purge those fabrics that you know for certain you won't use. That's the baby step you referred to.
The definition of scrap size depends on the person who holds the scraps. I've heard and read references to the definition including something as large as 1/8 yard to a 5" square to a 2" strip. A lot of it depends on what kinds of designs you like to make relative to the sizes of pieces you're willing to work with.
Enjoy your stash and take comfort that you belong to a very large club; most quilters are in the same boat as you.