I started out with books from the library. If your library doesn't have much you can request them to purchase quilting books and DVDs (probably need to be specific with your requests), or ask about inter-library loan.
http://www.smartflix.com rents DVDs on many subjects, including quilting. I haven't seen their DVDs, and the rentals are a little on the pricey side IMO; however, it would be a lot less expensive than purchasing a lot of DVDs.
Eleanor Burns "Quilt in a Day" books and DVDs are popular with beginning quilters. You might be able to find some of them on eBay or Amazon used.