I had a similar experience, but early on when they were looking for volunteers to demonstrate hand quilting at the Fair, I signed up. During that demo I got acquainted with 2 other old time members and from then on they were friendly. That friendship led to working on the Program Committee which broadened my acquaintances. So, I'd suggest you get involved.
I also understand about not quite fitting in as your first love is doll making. Just in the past year and a half we have started a "fiber artists" group in Tulsa which is a chapter of a long standing group in Oklahoma City which is about 100 miles away. We have basket weavers, doll makers, art quilt people, mixed media, weavers, dyers, any kind of fiber related activity. All of us feel like this is where we really belong and show and tell at our monthly meetings is always the highlight as you never know what is coming out of those bags. It's taken awhile but it started when we were entering pieces in a juried show in OKC and we began to find each other and pool our delivery and pickup. Keep your eyes and ears open for fiber shows in your area and other "fiber" people that you can "commune" with. Isn't there a top notch art museum in New Hampshire that caters to fibers?
Keep looking,
Jan