It's a matter of personal choice, I think. I have a fairly modest stash, but I use things there frequently. Most of the time, I buy fabrics I see that I love - things that would be the focal point of a quilt, but that I don't have a specific use or reason for at the time. I learned a long time ago that for me, at least, it's best to buy those fabrics that grab your attention when you find them, because when you decide you want it in the future, it is unlikely to still be available. My mother, on the other hand, buys fabric for each project, but nothing for a "stash" per se.
Neither way is wrong. It's all a matter of what works for your specific situation, financial constraint, or creative process.
