Earlier quoted context omitted.
Taboos are often controversial. And if he were a sociologist or anthropologist, published in the field with any reputation at all in it, who could speak on that complex dynamic over history and the interplay between environment, tradition, human biology, and social context, he might have a prayer of using reason to move the needle on the taboos in his own culture. But he's not. He's a professor in another field speak…
But does he have a right for his personal opinion in a free country?
But he's using that right to say very public things that are very objectionable, in a space he understands poorly, and everyone else can use their same rights to call him on his nonsense. And when one is an academic, one's word carries weight outside one's area of expertise.
For all the good he did for science education, a common criticism of Carl Sagan is he was an astrophysicist (damn good one) who dabbled in neurobiology, which was well outside his area of expertise---his oft-repeated "reptile brain" theory basically doesn't match to a contemporary understanding of neuro-anatomy and didn't when he wrote Cosmos either. But because he shared it from his platform and wrote a book on the topic, "humans are a fish brain wrapped in a lizard brain wrapped in a monkey brain" is an oft-repeated untruth.
We hold those whose reputations and positions are built on knowledge more accountable to be right when they speak than we hold others. Stallman chooses to exercise his freedom of speech, and we choose to hold him accountable for his position on topics that have real consequences for people who aren't him.