As a sanity check, http://books.google.com/books?id=GyE8Qt-kS1kC&pg=PA151&dq=&l... includes an estimate that approximately 10% of children in the 1700s died from smallpox. The first successful smallpox vaccine was made from cows. And the inspiration for the vaccine was the realization that the reason why milkmaids had good complexions was that something about their job gave them some level of protection against smallpox.
In mammals of all kinds, including humans and cows, milk delivers antibodies to protect babies. Therefore having fresh milk, and the fresher the better, provides protection from smallpox. This both helps survival and, as was apparent in the complexions of the milkmaids, resulted in more attractive adults.
There you have a very strong selection pressure to be able to drink fresh milk.