I used to buy on speculation and started with smaller amounts. Then I started buying some 2-3 yard pieces. Now, we are facing downsizing and I am working on using up the stuff I like but don't love. The stuff I love is my retirement stash. I feel comforted by having a certain amount of stuff stashed away but am getting to the point where I feel like I really do have too much and knowing that we'll be moving from a house to a two bedroom condo makes using it a realistic plan.
Oh, and I remember reading that Alex Anderson buys a third of a yard instead of a fat quarter. Considers that more useful.