The problem may be hard to notice or acknowledge for many people, but there's actually a lot of pressure put on people to socialize in some way and interact in some way, and that is mean. Being mean in return may not be the most effective reaction, but it is quite understandable.
To illustrate you how badly skewed it is: for a few years, me and my partner had a foster home for stray kittens. In Poland. That means lots of hard work that rarely pays off (by having a healthy kitten that gets adopted, there is no money in it, obviously), depressing amount of death and suffering, participating in such great activities as autopsies and interventions.
It was a significant part of our lives, our passion and when it paid off, it brought immense satisfaction. It was a very natural subject of talk for us, but really -- do you want to hear about the state of an inflamed heart we found in a dissected kitten, what happens to bodily functions when some dude flails a cat while holding it by the tail, how a neglected cat lost her eye to disease or other fascinating subjects from this category? All those conversations while eating?
Then why should people who aren't passionate about children endure conversations about children defecating? Really. I can handle it, of course (while I don't love it), but I can repay with a story about a dog eating her own intestines, so I'm probably not a benchmark for most sensitive listener.
People not wanting to hear about babies are just like people not wanting to hear about eyes falling out of sockets, it's only ours (yeah, I can talk about babies too) presuppositions that make the subjects seem so different.
(btw, if you rob me of my daily walk, I will fall asleep in your office)