DH's are usually right (grrrrr) but stressing yourself over a hobby is just not what a hobby is about.
Good decision. Do what you enjoy and your grandchildren will love you even more you are happy. They will never miss what they have not had.
So, go girl and enjoy a great hobby and have fun with those grandkids and build loads of happy memories for them.