We got crap from family, a few friends (good friends were fine about it) and, lots of strangers for whom it should have been none of their business.
Basically, when most people ask why, there's no answer other than infertility that results in anything other than them telling us we're wrong. So many people who have or want to have kids see voluntarily choosing not to have them as not just a personal choice, but as an affront to everything they believe in.
That's not even to mention those who want and can't have kids who have gotten really bitter when the topic comes up. I just avoid it whenever I can.
On the plus side, as we passed 40, I think people assume that we couldn't rather than we chose not to have kids and very few people ask anymore.
If you have or want kids and that makes you happy, wonderful. But, far too many people take what's a pretty longstanding decision as a challenge to convince us to change our minds.