I enjoy the entire quilting process, starting with fabric shopping! I don't do lots of other things, like eating out or going on expensive (or cheap) vacations. If someone who ate a nice dinner out once a week, instead spent the same amount on fabric, their stash would grow by leaps and bounds.
I think it's all what you think is important enough to spend your money on, and besides, fabric is non-fattening!