Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Semantics derived automatically from language corpora contain human-like biases

scim.ag

Post reply on HN
Print view View original

61–70 of 92 posts

Re: Semantics derived automatically from language corpora contain human-like biases

#61

Post by arrrg »

noir_lord wrote: post #60
anigbrowl wrote: post #54

Earlier quoted context omitted.

Why would we want to reproduce existing structures of oppression in mechanical form? Have you noticed how automation often vastly amplifies things? It's a short step from saying 'this model accurately reflects the bias in society' to 'that's how things are, the computer says women aren't cut out to be doctors.' Surely you are aware that in real world world people rationalize decisions they don't actually understand a…

> Why would we want to reproduce existing structures of oppression in mechanical form? If (for example) 66% of Doctors are male and 34% female then it's not reproducing "existing structures of oppression" it's inferring something about reality .

If it’s saying that and only that then that’s obviously fine. However, if that knowledge is then applied in any other way, then that’s problematic.

Re: Semantics derived automatically from language corpora contain human-like biases

#62

Post by Houshalter »

anigbrowl wrote: post #54
Houshalter wrote: post #49

Earlier quoted context omitted.

The distinction is very important. If it's just regurgitating human biases that would be bad. Humans often have very inaccurate and warped beliefs after all. If it's accurately modelling reality, then what's the problem? That's what we want it to do. Why would you want a less accurate model of reality? I've seen interpretations of this result that think it's proof "language is sexist" or whatever. But there's no evid…

Why would we want to reproduce existing structures of oppression in mechanical form? Have you noticed how automation often vastly amplifies things? It's a short step from saying 'this model accurately reflects the bias in society' to 'that's how things are, the computer says women aren't cut out to be doctors.' Surely you are aware that in real world world people rationalize decisions they don't actually understand a…

>structures of oppression

What oppression? How are word vectors oppressing anyone? What a ridiculous claim.

>Have you noticed how automation often vastly amplifies things?

No, not at all. I've heard this claim on similar discussions. But I've yet to see a convincing example. Particularly with word2vec. I find it very implausible that word vectors will somehow discriminate against female doctors or whatever.

>It's a short step from saying 'this model accurately reflects the bias in society' to 'that's how things are, the computer says women aren't cut out to be doctors.'

No it's not a short step at all. No one is ever going to use word vectors to figure out what genders are capable of what jobs. At worst, your auto-correct might be slightly less likely to suggest "doctor" for a misspelled word occurring in a female context. And on net it will still make more accurate corrections than the alternative.

Re: Semantics derived automatically from language corpora contain human-like biases

#63

Post by notahacker »

noir_lord wrote: post #60
anigbrowl wrote: post #54

Earlier quoted context omitted.

Why would we want to reproduce existing structures of oppression in mechanical form? Have you noticed how automation often vastly amplifies things? It's a short step from saying 'this model accurately reflects the bias in society' to 'that's how things are, the computer says women aren't cut out to be doctors.' Surely you are aware that in real world world people rationalize decisions they don't actually understand a…

> Why would we want to reproduce existing structures of oppression in mechanical form? If (for example) 66% of Doctors are male and 34% female then it's not reproducing "existing structures of oppression" it's inferring something about reality .

In an environment in which Blue people are banned from becoming doctors, its also inferring something about reality to conclude that 0% of Doctors are Blue. It would be entirely wrong, however, to use these inputs to infer anything whatsoever about the respective propensity of Blue and Green people to become doctors in an environment in which such a rule or idea of a rule had never existed. Obviously "structures of oppression" - real and imagined - which lead to fewer female doctors even in western liberal democracies where women wishing to become doctors are generally met with encouragement are less extreme, but that isn't to say they don't exist or that a computer output (or human interpretation of said computer output) is likely to draw correct inferences from it.

And if you think that people won't use the idea that the outputs are unbiased because the computer isn't programmed with the same prejudices that produce the inputs, I have some algorithmically-generated investment advice involving a bridge to sell you

Re: Semantics derived automatically from language corpora contain human-like biases

#64

Post by yummyfajitas »

dang wrote: post #42
yummyfajitas wrote: post #34

Positively right is a statement about beliefs. Normatively right is a statement about actions. It's my belief that if you want an ML system to take normatively right actions , you should explicitly encode the value of those actions (or the world states they are designed to achieve, if you are more utilitarian than virtue ethics) into it's utility function. To give a concrete example from the area of lending, you shou…

> because there are things man should not know I get that you like to take contrarian positions. But you also make a habit of inserting flamebait into your posts about them. This combination is trolling. If you continue to do this we will ban you. Specifically, we need you to stop playing the following game on HN: 1. Post contrarian view 2. Include provocation 3. People get provoked 4. Act like people can't handle yo…

You're wildly mischaracterizing what I said. I didn't suggest "people can't handle my truth". I suggested the article we are discussing says there are certain truths that ML systems should not learn.

The article explicitly recommends building systems which can't learn those things, and suggests characterizing them is a first step:

"We recommend addressing this through the explicit characterization of acceptable behavior. One such approach is seen in the nascent field of fairness in machine learning, which specifies and enforces mathematical formulations of nondiscrimination in decision-making (19, 20). Another approach can be found in modular AI architectures, such as cognitive systems, in which implicit learning of statistical regularities can be compartmentalized and augmented with explicit instruction of rules of appropriate conduct (21, 22)."

According to the article, the most closely related work "is concurrent work by Bolukbasi et al. (6), who propose a method to “debias” word embeddings."

(Recall that in the context of the article, "bias" is something that generates true predictions that are objectionable rather than something which is false.)

Re: Semantics derived automatically from language corpora contain human-like biases

#65

Post by dang »

yummyfajitas wrote: post #64
dang wrote: post #42

Earlier quoted context omitted.

> because there are things man should not know I get that you like to take contrarian positions. But you also make a habit of inserting flamebait into your posts about them. This combination is trolling. If you continue to do this we will ban you. Specifically, we need you to stop playing the following game on HN: 1. Post contrarian view 2. Include provocation 3. People get provoked 4. Act like people can't handle yo…

You're wildly mischaracterizing what I said. I didn't suggest "people can't handle my truth". I suggested the article we are discussing says there are certain truths that ML systems should not learn. The article explicitly recommends building systems which can't learn those things, and suggests characterizing them is a first step: "We recommend addressing this through the explicit characterization of acceptable behav…

I don't believe I mischaracterized anything; you did throw a gratuitous provocation in that comment (more than one in fact). Obviously, though, I'm not describing one comment but a longstanding pattern. If you want to continue commenting on HN, this needs to stop.

I've made many attempts to explain, don't believe anyone could accuse us of being impatient with you, and do believe you're more than smart enough to understand. If your account consistently produces troll effects on HN, which it does, then at some point it's you who are responsible—not people who can't handle the truth, don't want 'man' to know things, don't know math, or however else you blame others. At some point enough is enough.

If you really need a further explanation I'd be happy to try, but would need some indication that you're asking in good faith.

Re: Semantics derived automatically from language corpora contain human-like biases

#66

Post by scribu »

notahacker wrote: post #57
scribu wrote: post #46

Earlier quoted context omitted.

I agree that if your goal is to build a machine that decides who gets to become a doctor, you need to do more than just let it loose on a bunch of text. But I don't think preventing it from learning the current state of the world is a good strategy. Adding a separate "morality system" seems like a more robust solution.

A black box neural network attempting to draw inferences from a human-biased dataset - potentially even more biased because it can't understand subtexts - and then verifying that conclusion through an ad-hoc set of "morality checks" entirely independent from how it reached the conclusion sounds like a recipe for disaster. That's even before the marketing people get involved and start claiming the system is free from…

I'm not sure what your objection is with regards to the independence aspect. Why would having the morality checks integrated into the "learning about the world" part be better?

If you had an unwavering moral code which dictated that men and women should be treated equally, for example, why would it matter which facts are presented to you, in what order, or how you process them? Your morality would always prevent you from making a prejudiced choice, in that regard.

Re: Semantics derived automatically from language corpora contain human-like biases

#67

Post by sp332 »

EGreg wrote: post #15
randomwalker wrote: post #2

Coauthor here. Some of the press articles about our work didn't have a lot of nuance (unsurprisingly), but in the paper we're careful about what we say, what we don't say, and what the implications are. Happy to engage in informed discussion :)

Isn't the word bias being redefined by a social justice point of view? Normally bias would be with reference to failing to match reality (eg women in general have physically weaker upper body than men), and not failing to match whatever standard of equality a society wishes were the case eventually.

You should check out the Implicit Association Test that they used to measure the biases. Just as one example, there's nothing about being a doctor that is inherently more male or female. So all gender differences would have external causes.

Re: Semantics derived automatically from language corpora contain human-like biases

#68

Post by pharrington »

Houshalter wrote: post #53
pharrington wrote: post #47

Earlier quoted context omitted.

An exercise: Words relating to insects will occur in news articles about environmentalism, crop production, rituals of rebirth, etc. Words relating to plants might occur in articles about crop destruction, the international drug trade, people getting poisoned, etc. rmxt questioned the universality of sentiment analysis. Responding by noting specific contexts, free from a clear coherent general structure, is an assert…

But it is a universal truth that humans generally find plants pleasant and insects unpleasant. And the word "pleasant" is entirely based on human preferences after all.

What I'm probably missing indeed is that scoping of universality to humans. Lately I've been trying to be more explicit in my written communications in an attempt to understand both the limits of my knowledge and perceptions and the limits of the sources of information that I digest.

Is suggesting that pleasantness is a sentiment that's not unique to humans really that controversial?

super late edit: it's specifically flowers, not plants, that people are biased towards finding pleasant

Re: Semantics derived automatically from language corpora contain human-like biases

#69

Post by rspeer »

scribu wrote: post #58
rspeer wrote: post #56

Earlier quoted context omitted.

What do you think of Bolukbasi's approach that's mentioned in the article? In short, you let a system learn the "current state of the world" (as reflected by your corpus), then put it through an algebraic transformation that subtracts known biases. Do you consider that algebraic transformation enough of a "morality system"? I hope you're not saying we shouldn't work on this problem until we have AGI that has an actua…

> put it through an algebraic transformation that subtracts known biases > Do you consider that algebraic transformation enough of a "morality system"? I would consider it a sort of morality, yes. But keep in mind that the list of "known biases" would itself be biased toward a particular goal, be it political correctness or something else.

Yes, every step of machine learning has potential bias, we know that, that's what this whole discussion is about. Nobody would responsibly claim that they have solved bias. But they should be able to do something about it without their progress being denied by facile moral relativism.

If we can't agree that one can improve a system that automatically thinks "terrorist" when it sees the word "Arab" by making it not do that, we don't have much to talk about.

Re: Semantics derived automatically from language corpora contain human-like biases

#70

Post by nschucher »

Houshalter wrote: post #62
anigbrowl wrote: post #54

Earlier quoted context omitted.

Why would we want to reproduce existing structures of oppression in mechanical form? Have you noticed how automation often vastly amplifies things? It's a short step from saying 'this model accurately reflects the bias in society' to 'that's how things are, the computer says women aren't cut out to be doctors.' Surely you are aware that in real world world people rationalize decisions they don't actually understand a…

>structures of oppression What oppression? How are word vectors oppressing anyone? What a ridiculous claim. >Have you noticed how automation often vastly amplifies things? No, not at all. I've heard this claim on similar discussions. But I've yet to see a convincing example. Particularly with word2vec. I find it very implausible that word vectors will somehow discriminate against female doctors or whatever. >It's a s…

>No one is ever going to use word vectors to figure out what genders are capable of what jobs.

How can you possibly make this claim?

Biased word embeddings have the potential to bias inference in downstream systems (whether it's another layer in a deep neural network or some other ML model).

It is not clear how to disentangle (probably) undesirable biases like these from distributed representations like word vectors.

Post reply on HN