One thing that jumped out at me when I read your post was the word....he! Doctors are no different than husbands.
Just try going to a different specialist then he recommended - boy does that get their attention.
Back when I needed a slip to see another Dr. my primary told his nurse not to give one to me. He didn't know that I heard him. I went to the guy I wanted to anyway and the next time I saw my primary I suggested that things can get mixed-up when info isn't discussed directly so would he please tell me first. Twenty years ago and he always listens to me