I remember feeling that way at times long ago and in many cases by the time I got around to using a fabric I was saving I no longer liked it.
Now I use only what I love, which is most of my fabric. I don't save it for "someday". There will always be a new one I want to buy.