Yes, I agree with everyone who posted to find another doctor. Yes, doctors are stressed and have to see so many patients due to all the regulations but how else can a doctor correctly diagnose you w/o listening to your symptoms?
I agree that you may need to see an allergist and I would ask for a referral from his nurse.
All doctors are not like the one you described. also, you can ask for your medical records to take to a new doctor. That helps from starting over.