Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Semantics derived automatically from language corpora contain human-like biases

scim.ag

Post reply on HN
Print view View original

31–40 of 92 posts

Re: Semantics derived automatically from language corpora contain human-like biases

#31

Post by andreasvc »

Houshalter wrote: post #25

The paper and title implies it's absorbing these stereotypes from humans. I think there is another explanation. Remember these models are trained on a dataset of news or Wikipedia articles. And it's 'goal' is to find vectors that predict what contexts words are more likely to appear in. So if 34% of doctors are female, then you would expect 34% of doctors in news or Wikipedia articles to be female. Even if the articl…

I think you are making a distinction without a difference. If the word vectors pick up biases from wikipedia text, than for all practical purposes, they are (indirectly) absorbing stereotypes from humans. This is an expected result, but not necessarily desirable in the end.

Re: Semantics derived automatically from language corpora contain human-like biases

#32

Post by slackstation »

randomwalker wrote: post #2

Coauthor here. Some of the press articles about our work didn't have a lot of nuance (unsurprisingly), but in the paper we're careful about what we say, what we don't say, and what the implications are. Happy to engage in informed discussion :)

Since we think of biases of a large human corpora as wrong, I'm curious how one would find one or make one that is "right".

Given how accurate human corpora is at predicting things like gender distribution in jobs for instance, wouldn't making an "unbiased" corpora make an inaccurate AI?

Shouldn't we be careful in implying things like the biases and solutions to said biases? For instance, I'd like to know if my algorithm for filtering job applicants is trying to undo the injustices of the world in addition to finding the best candidates.

Re: Semantics derived automatically from language corpora contain human-like biases

#34

Post by yummyfajitas »

Positively right is a statement about beliefs. Normatively right is a statement about actions.

It's my belief that if you want an ML system to take normatively right actions, you should explicitly encode the value of those actions (or the world states they are designed to achieve, if you are more utilitarian than virtue ethics) into it's utility function.

To give a concrete example from the area of lending, you shouldn't build a bot that can't accurately infer that blacks don't pay back their loans as much as whites. You should instead assign an explicit dollar value to how much you want to lend to blacks, and maximize profit + that dollar value. I.e., "lending to a black person is worth $500 to me".

My takeaway from this paper is that bots accurately learn accurate stereotypes from language, and word embeddings do a great job of learning true facts about the world. But some people wish they didn't, because there are things man should not know.

Re: Semantics derived automatically from language corpora contain human-like biases

#35

Post by yummyfajitas »

andreasvc wrote: post #31
Houshalter wrote: post #25

The paper and title implies it's absorbing these stereotypes from humans. I think there is another explanation. Remember these models are trained on a dataset of news or Wikipedia articles. And it's 'goal' is to find vectors that predict what contexts words are more likely to appear in. So if 34% of doctors are female, then you would expect 34% of doctors in news or Wikipedia articles to be female. Even if the articl…

I think you are making a distinction without a difference. If the word vectors pick up biases from wikipedia text, than for all practical purposes, they are (indirectly) absorbing stereotypes from humans. This is an expected result, but not necessarily desirable in the end.

The parent's point is that they may not be absorbing stereotypes from humans at all. They may be generating accurate beliefs about the world from text representations of the world.

Re: Semantics derived automatically from language corpora contain human-like biases

#36

Post by scribu »

andreasvc wrote: post #31
Houshalter wrote: post #25

The paper and title implies it's absorbing these stereotypes from humans. I think there is another explanation. Remember these models are trained on a dataset of news or Wikipedia articles. And it's 'goal' is to find vectors that predict what contexts words are more likely to appear in. So if 34% of doctors are female, then you would expect 34% of doctors in news or Wikipedia articles to be female. Even if the articl…

I think you are making a distinction without a difference. If the word vectors pick up biases from wikipedia text, than for all practical purposes, they are (indirectly) absorbing stereotypes from humans. This is an expected result, but not necessarily desirable in the end.

The GP is saying that the bias isn't an attribute of the Wikipedia text, but of reality.

If the reality is that only 34% of doctors are female, why is it not desirable for the machine to learn that?

Re: Semantics derived automatically from language corpora contain human-like biases

#37

Post by andreasvc »

scribu wrote: post #36
andreasvc wrote: post #31

Earlier quoted context omitted.

I think you are making a distinction without a difference. If the word vectors pick up biases from wikipedia text, than for all practical purposes, they are (indirectly) absorbing stereotypes from humans. This is an expected result, but not necessarily desirable in the end.

The GP is saying that the bias isn't an attribute of the Wikipedia text, but of reality. If the reality is that only 34% of doctors are female, why is it not desirable for the machine to learn that?

Again, from humans and from reality is not different. Whether it is desirable to avoid stereotypes depends on your values.

Re: Semantics derived automatically from language corpora contain human-like biases

#38

Post by jrkatz »

scribu wrote: post #36
andreasvc wrote: post #31

Earlier quoted context omitted.

I think you are making a distinction without a difference. If the word vectors pick up biases from wikipedia text, than for all practical purposes, they are (indirectly) absorbing stereotypes from humans. This is an expected result, but not necessarily desirable in the end.

The GP is saying that the bias isn't an attribute of the Wikipedia text, but of reality. If the reality is that only 34% of doctors are female, why is it not desirable for the machine to learn that?

It depends on what you want the machine to do. If you are making a gambling machine that looks at pairs of names and makes bets as to which name belongs to a doctor, you want it to learn that.

If the machine looks at names and decides who to award a "become a doctor" scholarship to, based on who it thinks is most likely to succeed, you don't want it to learn that.

Re: Semantics derived automatically from language corpora contain human-like biases

#39

Post by dang »

draw_down wrote: post #7
reader5000 wrote: post #3

It's very amusing SJWs getting triggered by AIs.

Shut up.

Whoa, posting like this is a bannable offence on Hacker News. Your recent comment history looks pretty civil, so we won't ban you, but really please don't do this.

Re: Semantics derived automatically from language corpora contain human-like biases

#40

Post by rmxt »

yummyfajitas wrote: post #35
andreasvc wrote: post #31

Earlier quoted context omitted.

I think you are making a distinction without a difference. If the word vectors pick up biases from wikipedia text, than for all practical purposes, they are (indirectly) absorbing stereotypes from humans. This is an expected result, but not necessarily desirable in the end.

The parent's point is that they may not be absorbing stereotypes from humans at all. They may be generating accurate beliefs about the world from text representations of the world.

So, "plants are pleasant" or "insects are unpleasant" are universal truths?
Post reply on HN