It is expensive when you're trying to build up your stash. I use some of the shops Gal288 listed above. I also shop at Crafttown, StitchNFrame, Whittle's, etc for online savings. You will get a better quality of fabric if you look for clearance fabric in a quilt shop versus a big box store. You can find good fabric for occasionally $3-$7 a yard on sale. StitchNFrame sends out a weekly special on Wednesdays. You can google them because it's on special now through Fri morning. Crafttown has Last Call Fabrics that are usually marked down. Whittle's and Quilt Fabric Closeouts buy from distributors and are usually selling discontinued fabrics at really good prices.
My personal motto is that is if isn't on sale or clearance, I don't buy it. I look for the best quality for the cheapest price. At the end of the year, fabric shops will mark some of their slower moving fabrics down just to get rid of them before tax time too. I've gotten really good deals in December for the past two years.
I'm also a fairly new quilter (3 years) I first focused on solids and blenders because you can grab those into almost any quilt. Then I went to types of fabric that I seemed drawn to, i.e. Civil War, 1930s, batiks, etc. I've gradually increased my selections with landscape fabrics, novelties, tone on tone etc. The standard is usually to buy 3-4 yards of a focus fabric that you think you might like to make the main fabric of a quilt. Borders can be anywhere from 1-3 yards depending on the width of the border. 1 yard will almost always do a binding on any size, up to a king.
I never buy less than 1 yard. You will have to put your mark on this by selecting fabrics that will suit your need too. Will you be making mostly bed size quilts, baby quilts, wall-hanging or decorative quilts, etc.
Hope this helps some.