Hi Michael!
My name is Valerie and I am a fabric collector also. Let's be positive about this ... if your budget is above water, you can walk through your house, YOU ARE FINE!
You can't take it with you ... and quilter's love to share. If you are feeling out of control of your stash, that is another issue and will just take time sorting, folding, and creating. Possibly destashing via the site here or ebay.
Would we begrudge an water color or oil painting artist their paints, brushes etc? They don't own just one or two and often have to buy something new for a new project.
It is all in how you look at it. Have a great day!