Adding to the commute would be a huge issue for me. It takes time away from children and family. If you could move closer to work instead of farther away, the idea of the move would be much more appealing to me.
Agree also that things can go wrong in a new place. Since you have already put so much work and money into this one, I wonder if there are ways of adding landscaping/trees/fencing to improve privacy.
Another consideration is closeness to medical facilities, favorite doctors, etc. We drive a considerable distance now to see our family doctor, specialists, etc. because we don't want to change.
My husband couldn't stand the neighbors at our old house, and I have to admit they were pretty bad. However, now that we are out in the country living in his "dream" house, he is realizing that there are worse things than bad neighbors. For example, his normal commute of 45 minutes is sometimes 2.5 hours during the winter. There are other issues as well.