Semantics derived automatically from language corpora contain human-like biases

The paper seems to advocate for designing ML systems that learn that what is "true now" may not be "true forever and always". It seems to be quite the opposite of "there are certain truths that ML systems should not learn."

Various behavioral accidents can easily become embedded in culture, laws, and, yes, programs, at which point it stops mattering if they represent reality or "reality"; the real world will happily follow the cultural construction.

Sure, it's theoretically possible that an algorithm parsing text about medics' credentials that (e.g) positively weights male names and references to all-boys' schools and negatively weights female names and references to Girl Guides will be on average fair after an ad hoc re-ranking of all its candidates to take into account the instruction to treat male and female candidates equally. It's just unlikely to achieve this without completely reorganizing its underlying predictive model

[1]there's an interesting parallel to ongoing human arguments about how a machine should follow its "morality checks" should do this: does it ensure the subjects are "treated equally" in terms of achieving 50/50 gender ratio irrespective of the candidate pool (thus potentially skewing it massively in favour of the side with the weaker applicants), does it try to weight results so gender balance reflects historic norms (thus permanently entrenching the minority)? Or does it try to be "gender blind" by testing all its inputs for whether they're gender biased and normalising for or discarding those which are, which is basically learning everything again from scratch...

Directly, no. Nobody is going to go 'ah, word2vec - a new tool with which to perpetuate patriarchal capitalism, mwuhahaha'...probably. People are weird that way.

But indirectly they certainly will. How about NPC character generators in MMORPGs? Or chatbots on social networks? Stock characters in auto-generated romance novels? The possibilities are endless.

No doubt you will these examples are ridiculous, because you seem like a rigorous scientifically minded person who would be careful not to use data in inappropriate contexts, and who would try to discount cultural or emotional factors in making strategic decisions. But you are only as good at this as your own self-awareness and willingness to acknowledge the existence of implicit bias.

And many people are quite different from you and more easily or willingly allow their judgment to be shaped by representational stereotypes. Marketing people aim to confirm their audience's worldview very closely so that consumers will be willing to identify with the commercial prompt when it arrives. Politicians and yellow journalists routinely abuse statistics to grab people's attention. And so on.

I urge you to think more about this, and in more imaginative fashion. People are often surprised by the unexpected applications of technology employed by others.

> Our findings are also sure to contribute to the debate concerning the Sapir-Whorf hypothesis (17), because our work suggests that behavior can be driven by cultural history embedded in a term’s historic use. Such histories can evidently vary between languages.

No, not really. Linguistic relativity makes a claim about the direction of causation--from language to thought. This study does nothing to test the direction of causation, which is usually considered possible only with controlled experiments. This chicken and egg debate has been going on for the better part of the last century--it is not an easy problem.

> Our results also suggest a null hypothesis for explaining origins of prejudicial behavior in humans, namely, the implicit transmission of ingroup/outgroup identity information through language. That is, before providing an explicit or institutional explanation for why individuals make prejudiced decisions, one must show that it was not a simple outcome of unthinking reproduction of statistical regularities absorbed with language.

Once again, this was not an experiment capable of producing causal evidence. The authors have shown that human biases can be replicated by statistical learning of language corpora--admirable work, but nothing new here for Sapir-Whorf.