Grandma, I hope it is not rude, because my husband did the same thing for me! Make it clear that lunch or dinner is not on you. Perhaps you can word it in a way that since you're not buying them a meal that a gift is not expected.
If the shoe was on the other foot, I would not expect you to pay for my meal!
One thing to keep in mind.. I couldn't really socialize with many guests during my "meal time party"; only the people that were sitting right beside me and in front of me. There were only about 10 of us at my surprise party, which was just about right. I would be afraid if there were more people, your husband might not be able to mingle at a restaurant the way he would be able to in a "party" setting. But kids have parties all the time in restaurants, so why not you?!
Good luck!