Hi April, what a decision!
IMHO, I would seriously think about staying in your home. It seems like you've put a lot of money and work into your home. With the economy right now you will probably not be able to recoup the investment that you made.
As you said the neighbors have changed. With that said, the neighbors may soon change again. As for the neighbors try to be friendly but you don't need to be buddy buddy with them.
I would not want to add to my drive to work. Having more property (space) can mean more work. That work takes time away from the things you love to do and the people you want to be with. As for the privacy, you could invest in a fence, or just stay to yourselves. I wouldn't want a fence as it blocks the view of what is going on in the neighborhood. You have many memories in your home, you've made it how you like it and love it so I would stay put.
If you feel the need, then just get away for the weekends on little trips. Sometimes that's all that's needed to realize that you love your home and love coming home to it.
Good luck with your decision.
Pam M