I second shopping at thrift stores. I never go anywhere else if I can help it.
I use sheets for backing and binding. Sometimes I can score good sheets for under a dollar.
I buy cotton shirts with neat prints or plaid ones just dirt cheap.
My favorite thing is to buy hideous comforters, remove the fabric and use the inside for batting. I've already saved a small fortune on batting. Plus I can use the fabric to make test garments when sewing apparel or if it's not too bad I can make curtains out of it or something.