Maybe God is leading you to a new (old) place. If you husband feels so much happier serving people he needs to find a place where it is safe for him to do so. Maybe the move back "home" is the place God is calling you too.
I am also a Pastor's wife. However He (we) served 20 years in the Navy first then went thru Seminary and he is now one of the Pastor's at the church we had already been attending for 8 years