Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Notes on AI Bias

ben-evans.com

Post reply on HN
Print view View original

71–80 of 126 posts

Re: Notes on AI Bias

#71

Post by moomin »

gambler wrote: post #51

>The most obvious and immediately concerning place that this issue can be manifested is in human diversity. I swear, when someone starts building autonomous killer robots, the first set of concerned articles will probably be asking whether robots were properly trained to target all genders and races with equal accuracy. This is not a sensible way to approach AI ethics. >It was recently reported that Amazon had tried…

You’ve just read a long article that covers many aspects and zeroed in on your own hobby horse. You say there’s significantly bigger issues, but you don’t actually talk about that. Instead you talk about the thing you just said you didn’t think people should be talking about. There’s some serious projection going on here.

Re: Notes on AI Bias

#72

Post by michaelmior »

FakeComments wrote: post #66
michaelmior wrote: post #32

Earlier quoted context omitted.

Race and culture are two very different things. There certainly can be some correlation, but drawing conclusions of suitability for employment based on race is a problem.

I agree — race and culture are very different things. That’s why it’s only meaningful to draw these conclusions once you control for cultural variation. No one has shown that they drew conclusions based on race: they drew conclusions based on written words, which correlated with race. The claim is that this demonstrates racial discrimination. However, an equally valid explanation is culture as a confounding variable…

You're right, I didn't try to address it. I also didn't make any assumptions of racism in my comment. However, generally speaking, I don't think one culture is better than another either and discrimination based on culture is also a problem. (I understand that this is an opinion and many would disagree with me on that point.)

Re: Notes on AI Bias

#73

Post by meheleventyone »

DuskStar wrote: post #57
meheleventyone wrote: post #56

Earlier quoted context omitted.

The problem is that AI and more generally 'algorithms' are or were presented as neutral and unbiased. As such their biased results prop up a biased system. I don't think people are against using ML and for biased human systems. Just pointing out the ignorant, naive and lazy deference to computers that often occurs in human systems that share the same bias. In short I'd think most people who are against biased AI are…

Of course, sometimes reality is also biased, and the AI systems are just accurately reflecting reality. And that's an even bigger elephant.

I’m not sure what that even means if we know we can bias outcomes. Pretending there is some kind of natural state that is for the sake of being natural preferred seems odd given humans propensity to change the world to suit. I also suspect for many that ‘reality’ is really just a dog whistle for their preferred biases. Not to mention the entire issue with deriving and ought from an is.

Re: Notes on AI Bias

#74

Post by john-radio »

Bartweiss wrote: post #58
gambler wrote: post #51

>The most obvious and immediately concerning place that this issue can be manifested is in human diversity. I swear, when someone starts building autonomous killer robots, the first set of concerned articles will probably be asking whether robots were properly trained to target all genders and races with equal accuracy. This is not a sensible way to approach AI ethics. >It was recently reported that Amazon had tried…

That intern story is excellent; I'm adding it to my bank of "weird AI tricks" like pausing Tetris to avoid losing. More topically, you're quite right to object to that Amazon reference. As far as I can tell, the real story is even worse than mislabeling. Amazon devs wanted a system to spot candidates in resume banks, so they trained it to recognize resumes similar to the ones submitted to Amazon in the past. The enti…

> That intern story is excellent; I'm adding it to my bank of "weird AI tricks" like pausing Tetris to avoid losing.

Post your bank! Let's be like Magnus Carlson and occasionally ask ourselves, "What would DeepMind do?"

Re: Notes on AI Bias

#75

Post by cygaril »

theoh wrote: post #7
deogeo wrote: post #3

Earlier quoted context omitted.

Sounds like an extraordinarily poor AI system if it depends on absolute numbers, and not per capita. And wouldn't the number of unsuccessful hires also skew male?

Consider the possibility that the (pre-AI system) probability of success for a female applicant is the same as the probability of success of a male applicant. You could make a "per capita" quota as a kind of goal. That's not a problem, but how would you make sure the quota was met? The typical AI system doesn't work on the basis of selecting candidates entirely at random, pro rata, in order to meet a quota. It works…

From a machine learning point of view, one can just add the constraint that the probability of being in the "yes" bucket is that same for both male and female candidates. Doing this will give a worse fit than an unconstrained optimization, but it is fairer.

More sophisticated approaches are possible.

Re: Notes on AI Bias

#76

Post by DuskStar »

Bartweiss wrote: post #65
DuskStar wrote: post #21

Earlier quoted context omitted.

One situation I could see leading to this result (Amazon cancelling their resume filtering software with the excuse that it 'skewed male') is that 1. The AI system accurately predicted employee success across both genders AND 2. The AI system predicted that women would do worse than men That's politically embarrassing and something that you can't necessarily 'fix' by improving the system. (see: all the 'will this per…

In this case, it appears to instead be a matter of journalists focusing on totally the wrong aspect of a story for more drama. Buried deep in the original Reuters piece is this offhand mention: > Gender bias was not the only issue. Problems with the data that underpinned the models’ judgments meant that unqualified candidates were often recommended for all manner of jobs, the people said. With the technology returnin…

That is certainly another plausible explanation - and a less culture-war infused one, too. Thanks!

Re: Notes on AI Bias

#77

Post by joshuamorton »

gambler wrote: post #51

>The most obvious and immediately concerning place that this issue can be manifested is in human diversity. I swear, when someone starts building autonomous killer robots, the first set of concerned articles will probably be asking whether robots were properly trained to target all genders and races with equal accuracy. This is not a sensible way to approach AI ethics. >It was recently reported that Amazon had tried…

>Framing this problem as "bias"

Except that's exactly what it is. Much as your model was biased against interns.

> and especially hyper-focusing everyone's attention on diversity aspect of it is extremely irresponsible.

Why? Pointing out a specific and concrete harm badly designed ML models cause is irresponsible? Just because the same kind of methodological flaw can cause other harms its irresponsible to use a motivating example?

Re: Notes on AI Bias

#78

Post by gambler »

lonnyk wrote: post #70
gambler wrote: post #51

>The most obvious and immediately concerning place that this issue can be manifested is in human diversity. I swear, when someone starts building autonomous killer robots, the first set of concerned articles will probably be asking whether robots were properly trained to target all genders and races with equal accuracy. This is not a sensible way to approach AI ethics. >It was recently reported that Amazon had tried…

I get the point, but why didn't you just exclude intern resumes from the training data? Do you still suspect a skewed result?

>I get the point, but why didn't you just exclude intern resumes from the training data?

That was the logical next step and we started on that, but it required exporting more historic data out of the HR system and filtering out anyone who started as an intern as well. Sounds simple, but in practice it's anything but. Just for the reference, data extraction, cleaning and filtering in that project took at least an order of magnitude more time than anything related to machine learning.

The project eventually lost steam and got abandoned.

>Do you still suspect a skewed result?

Absolutely. My personal intuition is that there is very little correlation between resumes and candidate quality. If that is true, any seemingly accurate predictions would be the result of a similar problem. Testing this hypothesis was a large portion of why I agreed to work on the project in the first place.

Re: Notes on AI Bias

#79

Post by [deleted] »

layoutIfNeeded wrote: post #26
twa927 wrote: post #14

> Until about 2013, If you wanted to make a software system that could, say, recognise a cat in a photo, you would write logical steps. You’d make something that looked for edges in an image, and an eye detector, and a texture analyser for fur, and try to count legs, and so on, and you’d bolt them all together... I'm doing a lot of such algorithms (well, not for images). Does someone know if such algorithms have a na…

I would call it “classical” machine learning.

[deleted]

Re: Notes on AI Bias

#80

Post by gambler »

bduerst wrote: post #60
gambler wrote: post #51

>The most obvious and immediately concerning place that this issue can be manifested is in human diversity. I swear, when someone starts building autonomous killer robots, the first set of concerned articles will probably be asking whether robots were properly trained to target all genders and races with equal accuracy. This is not a sensible way to approach AI ethics. >It was recently reported that Amazon had tried…

If you train an AI using data from a system that already has certain biases, then the AI is going to replicate those same systemic biases in it's own predictions. It follows the "garbage in, garbage out" idiom. Curiously though, did you compare the non-hire (full time) rates of interns vs fire rates of non-interns?

>did you compare the non-hire (full time) rates of interns vs fire rates of non-interns?

Not sure I understand the question. IIRC, the way data was setup there was no way to tell why an intern stopped working for the company, because for all interns "reason code" for separation was the same.

Post reply on HN