He is working for YOU. I've gone through this before. You could address him directly and tell him your concerns first and give him a chance to change too but if it's ongoing and you're not comfortable, change doctors. I now have a doctor who listens to everything I have to say, then we discuss it and he lets me choose the course of action. He shocked me on my first visit, took my hands at the end of the appointment and asked if he could pray for me. He feels it is essential to his care for his patients and he is greatly loved for it.
Something else to remember is that doctors are given a certain amount of allotted time with a patient from the medical plans they accept and it's usually around 15 minutes. Mine doesn't end the appointment until I am satisfied. I've never felt pressured to hurry.