Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amazon scraps secret AI recruiting tool that showed bias against women

reuters.com

Post reply on HN
Print view View original

181–190 of 433 posts

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#181

Post by macinjosh »

zby wrote: post #159
macinjosh wrote: post #93

This will be unpopular but I don't care. What is the evidence that the source data for this 'AI' is biased because the men it came from did not want to hire women? Is there a reserve of unemployed non-male engineers out there? If so what evidence is there of that? Technical talent is both expensive and a rare commodity for tech companies. The non-male engineers I've worked with have always been exceedingly competent,…

"The non-male engineers I've worked with have always been exceedingly competent, smart, and their differing perspectives invaluable" - that is really an (anecdotal) evidence that there is a bias indeed. If the recruiting was all unbiased than the quality of existing male and female force would be the same - if female workers are of higher quality than it means that they need to pass higher requirements.

I didn't mean to imply the males I worked with weren't equally good. I was simply pointing out that, in my (anecdotal) experience, I couldn't see any reason non-males couldn't do the job well.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#182

Post by kareemsabri »

zby wrote: post #175
kareemsabri wrote: post #163

Earlier quoted context omitted.

Is the NBA biased against white guys?

I don't know - is it? What is the difference between bias and inferring information from skewed data?

Bias, to me, is the active (perhaps unconscious) discrimination based on a trait. Skew is an unequal distribution of that trait as a result of bias in favor of other traits, historical circumstances, or anything other than discrimination.

The NBA wants good basketball players. If they happen to be white, I imagine they'd draft them with equal enthusiasm as any other player. So no, it isn't.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#183

Post by zby »

macinjosh wrote: post #181
zby wrote: post #159

Earlier quoted context omitted.

"The non-male engineers I've worked with have always been exceedingly competent, smart, and their differing perspectives invaluable" - that is really an (anecdotal) evidence that there is a bias indeed. If the recruiting was all unbiased than the quality of existing male and female force would be the same - if female workers are of higher quality than it means that they need to pass higher requirements.

I didn't mean to imply the males I worked with weren't equally good. I was simply pointing out that, in my (anecdotal) experience, I couldn't see any reason non-males couldn't do the job well.

So everyone you haver worked with have always been exceedingly competent and smart?

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#184

Post by stcredzero »

fuscy wrote: post #25

The eye opening thing here is not that the AI failed, but why it failed. At start the AI is like a baby, it doesn't know anything or have any opinions. By teaching it using a set of data, in this case a set of resumes and the outcome then it can form an opinion. The AI becoming biased tells that the "teacher" was biased also. So actually Amazon's recruiting process seems to be a mess with the technical skills on the…

The same AI should be applied to hiring nurses and various other fields which show population skews in gender, as well as fields which are not skewed. I'd be curious as to the outcome.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#185

Post by Bartweiss »

sanxiyn wrote: post #38
michaelt wrote: post #29

Earlier quoted context omitted.

Could this be a direct indicator of a powerful subconscious bias in Amazon's existing hiring process? Maybe - but maybe not. Imagine a company with 2 men in HR, 2 women in HR, 40 men in engineering, and 10 women in engineering. That's with gender-blind hiring, reflecting only the 4:1 ratio of male to female CS graduates. If you picked a random male hire, there's a 40/42=95% chance they're an engineer whereas if you p…

Why would one train on "all hires CVs"? It'd be "engineering CVs", moreover it'd be "engineering applicants CVs", not "engineering hires CVs".

In the article I noticed: "Problems with the data that underpinned the models’ judgments meant that unqualified candidates were often recommended for all manner of jobs"

That language is a bit ambiguous, it could just mean that the algorithm failed on a wide variety of jobs beyond engineering. But another reading suggests that the algorithm was not asked "is this person a good fit for this role" but instead "what, if anything, is this person qualified for?"

If that's the case, then the problem starts to make more sense: the algorithm learned a correlation between male-sounding resumes and being hired for engineering roles. That could produce a biased approach even if the decisions in the training data were gender neutral but position-specific. Of course, it would also mean that an Amazon ML team trained an algorithm with inputs that didn't match to its eventual task, and makes me wonder what they used as a test set...

(Anecdotally, Amazon spent quite a while recruiting me for SysEng work I'm wildly unqualified for and uninterested in, even suggesting a switch to applying for that team when I was already in the funnel for something I'm more qualified at. When my resume eventually made it to a syseng engineer, they were rightly baffled that I had landed on in their stack, giving me the sense that something was screwy with how Amazon decides who heads towards which role.)

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#186

Post by izzydata »

briandear wrote: post #169
izzydata wrote: post #57

It is no secret that more men pursue careers in this field. How can you expect any algorithm to produce an equal number of men and woman applicants? If it did this then that would be actually biased in favor of woman. If this is what they want then they have to feed their neutral algorithm pre-biased data to get their expected results.

I think it isn’t equality in numbers but in activity screening out women. So there may still be a 8:1 ratio of men:women but according to the article, it would seem than even that 1 women would have been negatively impacted. I am not arguing the validity of their conclusions, just saying that I don’t think that it’s about equal numbers: given a male and female applicant, according to the article, women would have had…

Fair enough. I'm still skeptical of their interpretation without more details, but anything is possible.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#188

Post by mcguire »

exo762 wrote: post #11
trustmath wrote: post #2

I hate this industry. Shooting themselves in the foot over and over again because no one can get passed the idea that possibly, women can be just as good at math, logic and computer science - if people would just let them. This never ends. It's just one place after another, when it gets discovered. It never changes.

Why not "women are just not as interested in math, logic and computer science to pursue it AS OFTEN as men"? Why are you not considering this possibility?

Let's say you have two otherwise-identical resumes in front of you.

* One says "executed concentration camp prisoners in Kosovo". (Yeah, ok, I'm kidding. How about "Executed a plan to reduce production costs by 30%"?)

* The other says, "Won the Women's World Chess Championship 3 years in a row".

The first has five stars (thanks to "executed") and the second has three (courtesy of "women's"). Which are you more interested in?

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#189

Post by mcguire »

21 wrote: post #66
komali2 wrote: post #27

Earlier quoted context omitted.

Ah, the Damore argument. Besides the fact that his psuedo science has been summarily handled[0], to consider his argument you then have to equally consider the possibility of sexism in academia pressuring women to not study these subjects and societal pressure their whole lives pressuring them to not persue these career paths. There's also the idea that lack of women scientist "heroes" can be limiting (lack of role m…

> There's also the idea that lack of women scientist "heroes" can be limiting (lack of role models) This one is a bit weird, computer guys were always "nerds" and "geeks" to stay away from.

That kinda stopped being the case when geeks and nerds started making 6-figure incomes.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#190

Post by tomp »

theptip wrote: post #125
tomp wrote: post #81

Earlier quoted context omitted.

> The lower certainty would in turn lead to lower rankings for women even without any bias in the data. I don't think that's true. "No bias" means that gender is irrelevant (i.e. its correlation with outcome is 0%). Therefore the system shouldn't even take it into account - it would evaluate both men and women just by other criteria (experience, technical skills, etc), and it would have equal amounts of data for both…

> "No bias" means that gender is irrelevant False. If we're talking about the technical statistical definition, bias means systematic deviation from the underlying truth in the data -- see this article by Chris Stucchio with some images for clarification: https://jacobitemag.com/2017/08/29/a-i-bias-doesnt-mean-what... "In statistics, a “bias” is defined as a statistical predictor which makes errors that all have the…

I think the comments I replied to mean bias as in “sexist bias”.
Post reply on HN