Sometimes I feel exactly the way you describe. I justify (to myself) my quilting habit by knowing that I need a hobby and I love this one. It keeps me "off the streets and out of the bars". Over many years, I have invested a lot of $$ in equipment & supplies & fabric ... I never have or want more than a medium-sized fabric stash. I don't plan to buy any more fabric or quilting notions for a long time; I have plenty of stuff already and am working on several long-term quilt projects. Last year, my hobby budget was spent on machine embroidery designs (for quilts, mainly) until I now have more than I will ever use; this year, my hobby budget is focussed on machine embroidery thread, enough colors & yards to last me the rest of my life. (I have no plans as yet for my 2011 hobby budget.) None of my children are married or have children of their own. They very much like my quilts &, whenever I kick the bucket, they will have 1st claim on all sewing equipment, supplies, quilts, etc. They understand that what they do not want is to go to the ladies who make quilts for a specific children's home in this state.
I figure I can beat myself up about the expenditure of time & money, but the truths are this: (a) there are people who want & need ALL the quilts we quilters make and (b) I worked hard my whole life, even as a child; for decades, I did without everything but the necessities of life. Now that I do have the resources & time to have a hobby I enjoy, I'm going for it!
I surely hope you can ease your mind about your quilt making!