Since I only purchase fabric I really like for specific projects, I purchase at least 2 -3 yards. Also, I select other fabrics I like and purchase a couple of yards for my stash at the same time. As my stash builds I purchase colors I am missing. Currently I am adding browns and backgrounds in colors other than white, adding prints.
While I don't particularly like pastels, for our guild challenge project this year I have pastels, so on a recent roadtrip I actually found several I liked and purchased extra for my stash.
No matter how big my stash is, I always find several new pieces to add to it. Once you start, it will be easier. It is alot like eating potato chips.