Live data from Hacker News

Amazon scraps secret AI recruiting tool that showed bias against women

reuters.com

251–260 of 433 posts

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#251

One of the first good critiques of the "oncoming AI apocalypse" that I've seen is exactly this: the encoding of existing biases into the resulting AI system. But, it also means that bias is now measurable... ...and there were more than a few papers at NIPS that were directly dealing with "fairness" in a NN, aimed at addressing and using these issues and effects.

Weapons of Math Destruction offers good discussion on this problem, if you want to read more about it.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#252
post #220

Earlier quoted context omitted.

No body thinks that. You're way outside the overton window. Why is the health care field heavily biased in favor or female nurses and doctors? Are women smarter than men when it comes to biology/anatomy?

You're right, nobody thinks men are smarter than women. And it isn't true. So, let's think about why we see gender roles in employment. Why are there so few women software engineers? One possible explanation is that women just aren't smart enough. If you don't believe that (and I don't), then you need another explanation. Maybe it's because of sexism. But if you don't want to believe it's sexism (as the OP implied),…

> nobody thinks men are smarter than women

Actually I grew up hearing exactly the opposite - that girls are smarter than boys and that girls "mature" faster than boys.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#253

Earlier quoted context omitted.

> The number of women and men in the data set shouldn't matter (algorithms learn that even if there was 1 woman, if she was hired then it will be positive about future woman candidates). This is incorrect. The key thing to keep in mind is that they are not just predicting who is a good candidate, they are also ranking by the certainty of their prediction. Lower numbers of female candidates could plausibly lead to low…

How did you control for these things? Wondering what patterns there are that people use to prevent social discrimination. Seems challenging since much of AI, especially classification, is essentially a discrimination algorithm.

There are a few ways you can tackle this issue: 1) have the same algorithm for each group, but train separately (so in the end you have two different weights); 2) over-sample the group under represented in the data; 3) make the penalty more severe for guessing wrongly on female then male applicants during training; 4) apply weights to gender encoding; 5) use more then just resumes as data.

This isn't an insurmountable problem, but does require extra work then just "encode, throw it in and see what happens".

Amazon only scrapped the original team, but formed a new one in which diversity is a goal for the output.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#255

This will be unpopular but I don't care. What is the evidence that the source data for this 'AI' is biased because the men it came from did not want to hire women? Is there a reserve of unemployed non-male engineers out there? If so what evidence is there of that? Technical talent is both expensive and a rare commodity for tech companies. The non-male engineers I've worked with have always been exceedingly competent,…

Oh, there are a bunch of us, even here in the SF Bay Area. Trouble is, we're older than 35, or don't have degrees from "top" schools, and/or don't have the "passion" for bizarre extended hiring rituals. I could staff an entire dev team with non-male people within a week.

> we're older than 35, or don't have degrees from "top" schools, and/or don't have the "passion" for bizarre extended hiring rituals

This hits home so hard.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#256
post #220

Earlier quoted context omitted.

No body thinks that. You're way outside the overton window. Why is the health care field heavily biased in favor or female nurses and doctors? Are women smarter than men when it comes to biology/anatomy?

You're right, nobody thinks men are smarter than women. And it isn't true. So, let's think about why we see gender roles in employment. Why are there so few women software engineers? One possible explanation is that women just aren't smart enough. If you don't believe that (and I don't), then you need another explanation. Maybe it's because of sexism. But if you don't want to believe it's sexism (as the OP implied),…

> And that's where hands come up empty.

Maybe anti-male sexism prevalent in the health care and education fields is causing women to prefer those fields.

Fix the sexism in health care/education. Elementary teachers should be 50% men. Nurses should be 50% men. Instead those fields are 90%(!) women! That is a HUGE level of bias and discrimination

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#257
post #199

Earlier quoted context omitted.

Just being female is a qualification at your workplace?

a rather curious question... what prompted it?

The parent post? Evidently, for whatever reason, male applicants and coworkers are a problem for their organization, so they go out of their way and invest in "additional resources" to hire female candidates from the hiring pool. They're not hiring on merit, but on gender.

How exactly do you hire for female candidates without discriminating against the "overwhelming" body of male applicants? I would be really interested to know how this goes on behind the scenes: do you have open positions but cherry pick female candidates while disregarding male candidates from the get-go?

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#258
post #179

Stupid question probably but why not have two models.

Because evaluating differently based on gender actually is discriminatory .

Well, that sounds about right, then - every proposed solution to subtle, unconscious bias inevitably turns out to be explicit bias in the other direction.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#259

Earlier quoted context omitted.

> if people would just let them. People do let them. You can't force what people are interested in and you can't let in that which does not exist. In fact many places in tech give preference to women applicants, because they don't apply often and the companies want more women. They're just rare to see. :( There's no grand conspiracy. The truth is much less exciting: Women and men have different preferences, generally…

Please read the article. This article is about automated reasoning that discards resumes that are strongly correlated to resumes of women.

I think you need to read the comment again. The author's reply was to your comment saying "if people would just let them", not what the ML algorithm does in the article.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#260
post #176
post #27

Earlier quoted context omitted.

Ah, the Damore argument. Besides the fact that his psuedo science has been summarily handled[0], to consider his argument you then have to equally consider the possibility of sexism in academia pressuring women to not study these subjects and societal pressure their whole lives pressuring them to not persue these career paths. There's also the idea that lack of women scientist "heroes" can be limiting (lack of role m…

Where was it disproven? The article says that the research was controversial, not that it's false.

True or false isn't necessarily something I think you could say in debates about human genetics, yet.

For now I say it was "handled" in that not only did he fail to demonstrate that female disinterest in engineering, compared to male, is due to inherent psychological differences, and I quoted a couple people far more qualified than me that reached the same conclusion (their statements are in the article. The Wikipedia page is another good summary)

Post reply on HN