Live data from Hacker News

Amazon scraps secret AI recruiting tool that showed bias against women

reuters.com

91–100 of 433 posts

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#91
post #11

Earlier quoted context omitted.

Why not "women are just not as interested in math, logic and computer science to pursue it AS OFTEN as men"? Why are you not considering this possibility?

Because that's an inconvenient truth that doesn't fit the oppressor vs oppressed narrative.

[deleted]

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#92
post #67

I wish we could move away from resumes for tech role screening anyway, since they convey very little real reliable information. I’ve seen too many great hires from candidates with relatively weak resumes, and failed interviews from candidates with great resumes (and obviously vice versa). I’m not sure what the best alternative should be, though. I am a fan of open source work as a sort of code portfolio, but it doesn…

Unfortunately using open source work, even if only for programming, introduces all sorts of biases as well. A lot of very competent programmers work at jobs that do not have open source contributions and also have families which limit the time they can spend coding after work.

Good point! I didn’t mean to propose it as a solution, but rather as an enumeration or one possibility I’ve considered, and ruled out because it doesn’t work for everyone.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#93
This will be unpopular but I don't care. What is the evidence that the source data for this 'AI' is biased because the men it came from did not want to hire women? Is there a reserve of unemployed non-male engineers out there? If so what evidence is there of that?

Technical talent is both expensive and a rare commodity for tech companies. The non-male engineers I've worked with have always been exceedingly competent, smart, and their differing perspectives invaluable. If there was an untapped market of engineers you'd better believe every tech company would be taking advantage of it.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#94
“Instead, the technology favored candidates who described themselves using verbs more commonly found on male engineers’ resumes, such as “executed” and “captured”

This is the most interesting part to me. It’s suggesting that men and women think about things differently. Which, if true, suggests a lot of other possibilities that have relevance when hiring for certain positions.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#95

I would guess that the training data for the ML set was the set of all resumes and an indicator of whether the candidate was eventually hired (maybe with supplemental data about how far in the process the candidate got). Could this be a direct indicator of a powerful subconscious bias in Amazon's existing hiring process?

Right, so the "AI" is not finding the best candidates, but instead finding candidates that succeed in the old hiring process. What I'd like to see (though I'm sure this would get me laughed out of a manager's office) would be a certain fraction of random hires. Then we could train the model on success statistics based on 60,90,120 day performance (or longer!).

At the least, they should interview a certain subset of randomly chosen applicants or else the feedback loop from the interviewing process and the AI is going to grow tighter and tighter.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#96
I know diversity in tech is a hot button issue but I think hidden underneath the politics of it all the most important reason to champion more diversity is AI.

I don't think we're anywhere close to general AI so any AI system out there is built on what we feed it. If your music suggestion algorithm has a biased AI that's one thing, but when you're using AI to make critical decisions in society like who gets hired/recurited, medical diagnoses and other things you need to be extremely careful.

When designing AI models the broadest set of viewpoints should be considered. Any piece of AI is simply a reflection of it's creators, we need to make sure that some sort of equitable consensus is reached before deploying AI to critical human scale issues.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#98

What would happen if you removed gender from all HR and recruiting systems and then retrain the AI? Or for that matter, remove ethnicity, age, creed, etc... Is there any reason we need to be more specific? Race: Human Gender: Yes Age of legal contractual consent: Yes

The AI did not have access to gender. It was just word weighting and it turned out that words that could be linked to females ended up with applicants that had a negative outcome. Like the article says the AI ended up giving a negative weight to any resume containing the word women's as in women's [---] club , or those that mentioned certain all women's colleges.

It should be fairly easy to filter / replace all of that. The same logic can apply. Can we add some simple filters and re-train it?

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#99
It seems that the only socially acceptable output for the AI would have been hiring women 50% or more and hiring minorities at a rate greater than or equal to their representation in the populations. Anything else is clear bias and discrimination.

The project was doomed from the start.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#100
post #27
post #11

Earlier quoted context omitted.

Why not "women are just not as interested in math, logic and computer science to pursue it AS OFTEN as men"? Why are you not considering this possibility?

Ah, the Damore argument. Besides the fact that his psuedo science has been summarily handled[0], to consider his argument you then have to equally consider the possibility of sexism in academia pressuring women to not study these subjects and societal pressure their whole lives pressuring them to not persue these career paths. There's also the idea that lack of women scientist "heroes" can be limiting (lack of role m…

Population wide effects MLK dreamed about too. Reality is a different story. I think the whole end goal is misguided and is going to lead to a whole lot of frustration and disappointment and divisiveness.

Helping individuals to overcome biology is much simpler than doing it at population scale.

Post reply on HN