Oh hard. I would say to be direct. Tell them that since you are in front of the group more now, you have noticed the behavior and ask that they have respect for the speaker. If you do it at a time when the behavior is not happening, then you will not be admonishing anyone is particular. They may not be so defensive.
In our group we do ohhh and ahhh and make comments to our neighbors. And a few do carry on conversations every once in awhile. But it is in a room where the sound doesn't carry so that helped.
I just wanted to tell you I am sorry you have to deal with this and commisserate on that this is hard on you.