I read about this in "More" magazine lately and found this online. Apparently there are a lot of female hormones in milk to begin with, and with modern practices of milking cows while they are pregnant, it adds in even more hormones..
http://www.news.harvard.edu/gazette/.../11-dairy.html
It goes on to say most of the hormones are in the fat part of the milk.
"The milk we drink today is quite unlike the milk our ancestors were drinking" without apparent harm for 2,000 years, she said. "The milk we drink today may not be nature's perfect food."