Mine's pretty good. Although he doesn't understand why I "cut up fabric into little pieces just to sew it back together", he supports my love of it. He does the finances so, before I go spend a bunch on fabric, I usually check with him beforehand to make sure the budget can tolerate it. Either that or I save a dollar or two at a time until I have a bit of cash saved up.
I've found, in the 17 years that we've been married, that communication helps us avoid a lot of arguments. I would suggest calmly sitting down with your DH and asking him honestly why your quilting upsets him so much. There may be something bothering him or stressing him out that has nothing at all to do with quilting. Getting things out into the open is a step toward solving them. Hidden wounds tend to fester while those exposed to the light tend to heal.