I got addicted to buying fabric for a while when I'd been quilting for about a year. At first I bought mainly theme fabrics, then I started buying background fabrics. Now I have a nice stash and tons of scraps. I've been quilting about 2 1/2 years and just finished my 30th quilt. I have about ten in the hopper and I sew several days out of the week. I haven't been in a fabric store for months, however. I have all the fabric I can store.
I think whenever one starts a new passion, there are expenses to get up and running. I spent tons outfitting my kitchen when I got interested in cooking. I spent a lot when I got passionate about photography. And I spent a lot when I got passionate about quilting. After a while though, there isn't anything left that you need and the buying frenzy stops. I don't need anything for any of these hobbies now except specific items for a specific project--and even those things are few and far between.
If you buy so much fabric that there's no longer room to live in the house, or if you spend so much that you break the bank, that's the time to start worrying.