Amazon scraps secret AI recruiting tool that showed bias against women • Hacker News Board

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#381

Post by rrcaptain » Thu, Oct 11, 2018, 2:43 AM UTC

macinjosh wrote: post #93

This will be unpopular but I don't care. What is the evidence that the source data for this 'AI' is biased because the men it came from did not want to hire women? Is there a reserve of unemployed non-male engineers out there? If so what evidence is there of that? Technical talent is both expensive and a rare commodity for tech companies. The non-male engineers I've worked with have always been exceedingly competent,…

Biases are rarely knowingly done, that's why they're biases. The proof is in the pudding though. The model Amazon came up with was biased against women. That suggests that female candidates in the dataset were discriminated against.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#382

Post by c3534l » Thu, Oct 11, 2018, 2:48 AM UTC

megy wrote: post #374

c3534l wrote: post #370

I'm honestly pretty shocked that Amazon went ahead with this which is such an obvious legal liability in an extremely touchy area of business practice. Everything about this should have been a red flag that meant it never should have gotten off the ground, let alone been in place for years.

You don't think they should try new things, even though they might not be successful?

When it systematically and illegally oppresses a class of people in an obviously ethically dubious manner, I would expect a certain level of caution.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#383

Post by roenxi » Thu, Oct 11, 2018, 2:55 AM UTC

brown9-2 wrote: post #359

roenxi wrote: post #318

Earlier quoted context omitted.

Do you have some information not present in the article? There seem to be some assumptions on the training process in your comment that are not sourced in the article. I'll don my flack jacket for this one, but based on population statistics I believe a statistically significant number of women have children. A plausible hypothesis is that a typical female candidate is at a 9 month disadvantage against male employees…

Does Amazon give 9 months of parental leave, or are you saying women employees are disadvantaged for their entire pregnancy?

Ah. Sorry, silly me. A quick search suggests 20 weeks, so ~4.5 months.

Obviously I don't have much specific insight, so maybe there is a culture where they don't use leave entitlements. But if there are indicators that identify a sub-population taking a potentially 20 week contiguous break it is entirely plausible that it would turn up as a statistically significant effect in an objective performance measure. All else being equal, then a machine learning model could pick up on that.

The point isn't that it is the be-all and end all, just that the model might be picking up on something real. There are actual differences in the physical world.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#384

Post by rrcaptain » Thu, Oct 11, 2018, 3:02 AM UTC

gfodor wrote: post #164

darawk wrote: post #138

Earlier quoted context omitted.

Ok, so if you're hiring professional arm wrestlers, and your model looks at bicep muscle mass, is that discrimination because it selects against women? If you're hiring therapists, and your candidates take a personality test, and your ML model weights the 'nurturing' feature highly, is that discrimination because it selects against men?

Separate it one step further -- hiring decisions are all too easy to pin as discriminatory. What if a site shows ads for a special offer on protein shakes for people with higher bicep mass since they are in the target market? Is that discriminating against women?

How the hell are you targeting ads with "bicep mass"?

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#385

Post by rrcaptain » Thu, Oct 11, 2018, 3:15 AM UTC

hahan wrote: post #336

Call me cynical but I found it amusing that no one points out the fact that engineering work is laborious and dry to say the least for most people. That's the reason why there are so few people who have other options, females, upper middle class people, people of means, in the scene. There are so many lowish paid low status unsought for sectors where majority workers are male, say janitors in Universities, why I neve…

You don't see discussion on that because you're on a tech industry website, not a janitorial forum. And the percentage of women in janitorial work is nearly twice that as the percentage in tech per BLS: https://www.bls.gov/cps/cpsaat11.htm

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#386

Post by [deleted] » Thu, Oct 11, 2018, 3:26 AM UTC

ramblerouser wrote: post #294

dx87 wrote: post #271

Earlier quoted context omitted.

If it's anything like conferences trying to increase diversity numbers, they don't specifically target one gender, they focus their recruiting efforts on sites and locations that typically have a much higher representation, like a girls college, or a girls hacker group. That way they aren't actively saying "we won't accept male applicants", but they ensure that 90% of the applicants will be from the target demographi…

>We did eventually find a few great female applicants, but it took a lot of work and a lot of time dedicated specifically to that goal. How is this not sexism? Replace female with male in the above quote and tell me it isnt sexist.

[deleted]

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#387

Post by Yoyoyou » Thu, Oct 11, 2018, 4:23 AM UTC

fuscy wrote: post #25

The eye opening thing here is not that the AI failed, but why it failed. At start the AI is like a baby, it doesn't know anything or have any opinions. By teaching it using a set of data, in this case a set of resumes and the outcome then it can form an opinion. The AI becoming biased tells that the "teacher" was biased also. So actually Amazon's recruiting process seems to be a mess with the technical skills on the…

It failed because rationally interpreting gender data leads to politically incorrect conclusions.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#388

Post by 8note » Thu, Oct 11, 2018, 4:26 AM UTC

ewjordan wrote: post #361

lalaland1125 wrote: post #356

Earlier quoted context omitted.

> It wouldn't do this just because there were fewer women in the batches, rather it would just not assign any weight to those factors. That's exactly the issue we are talking about here. Woman's colleges would have less training data so they would get updated less. For many classes of models (such as neural networks with weight decay or common initialization schemes) this would encourage the model to be more "neutral…

I don't think I'm with you. A neural net cannot do this - picking apart male and female tokens requires a signal in the gradients that force the two classes apart. If there's no gradient, then something like weight decay will just zero out the weights for the "gender" feature, even if it's there to begin with. Confidence wouldn't enter in, because the feature is irrelevant to the loss function. A class imbalance does…

supposing waiter and waitress are both equally qualifying for a job, and most applicants are men, won't the ai score waiter as being more valuable than waitress?

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#389

Post by hamandcheese » Thu, Oct 11, 2018, 4:31 AM UTC

ramblerouser wrote: post #294

dx87 wrote: post #271

Earlier quoted context omitted.

If it's anything like conferences trying to increase diversity numbers, they don't specifically target one gender, they focus their recruiting efforts on sites and locations that typically have a much higher representation, like a girls college, or a girls hacker group. That way they aren't actively saying "we won't accept male applicants", but they ensure that 90% of the applicants will be from the target demographi…

>We did eventually find a few great female applicants, but it took a lot of work and a lot of time dedicated specifically to that goal. How is this not sexism? Replace female with male in the above quote and tell me it isnt sexist.

Diversity is generally accepted to be a positive thing. Targeting women specifically furthers that goal. Targeting men specifically furthers no goal.

So yes, sexism is not symmetric. Same with race.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#390

Post by hamandcheese » Thu, Oct 11, 2018, 4:35 AM UTC

petsormeat wrote: post #323

jhfhhhf wrote: post #287

Earlier quoted context omitted.

So what do you do now? Btw some men in tech are also over 35 and tired of hiring rituals.

After 15 years of front-end dev, I now work in retail. Some of my other peers are scraping by with Uber/Lyft. Some are muddling through as housewives or substitute teaching. And, yes, Bay Area tech hiring is needlessly hostile for men over a certain age as well.

I’m a bit confused... surely going through a needlessly bizarre hiring ritual is worth it compared to driving for Uber or being a substitute teacher?