So sorry that quilting is putting a kink in the works. I am sure that he just wants to spend more time with you.
My husband is ok with me sewing but still gets a little jealous that I am not spending time with him. So I wait until he has some kind of paperwork or bills to write or errands to run and then I do my sewing. Makes of both feel better.
Good luck and keep sewing/quilting.