I’m curious: how is it that this submission has not “reused” the submission I made eight hours before? In my experience, sending a link that is already there simply upvotes the existing submission.
[deleted]
Amazon scraps secret AI recruiting tool that showed bias against women
21–30 of 433 posts
Re: Amazon scraps secret AI recruiting tool that showed bias against women
#22I hate this industry. Shooting themselves in the foot over and over again because no one can get passed the idea that possibly, women can be just as good at math, logic and computer science - if people would just let them. This never ends. It's just one place after another, when it gets discovered. It never changes.
Why not "women are just not as interested in math, logic and computer science to pursue it AS OFTEN as men"? Why are you not considering this possibility?
That's always the excuse but an algorithm that shows bias against women has nothing to do with who has what interest unless that's a variable included in the data set initially. You are inferring that relation implicitly due to the result of the algorithm, but it doesn't mean that value is measured in the original set of data. If the algorithm is skewed to imply that, there is at least the possiblity that the algorithm has been trained to yield that result.
I wouldn't know unless I looked at all the data. But I'm not going to default to the popular opinion because that's literally half or more of the problem.
Re: Amazon scraps secret AI recruiting tool that showed bias against women
#23This is a direct and clear example of bias which made it easy to flag the ML algorithm. But what about ML algorithms that are inducing benefits to groups in less obvious contexts? What about groups that are not so easily identified as being protected classes by simple, human-understandable model features? What about cases where the features are just merely correlated with a subpopulation of a protected class? If we'r…
> What about cases where the features just merely correlated with a subpopulation of a protected class?
This is just Simpson's paradox [1] which is notoriously hard to identify because you have to compare the overall with the breakdown. As you say, current-AI probably already has such biases.
Re: Amazon scraps secret AI recruiting tool that showed bias against women
#24I hate this industry. Shooting themselves in the foot over and over again because no one can get passed the idea that possibly, women can be just as good at math, logic and computer science - if people would just let them. This never ends. It's just one place after another, when it gets discovered. It never changes.
Is this not Amazon trying to not discriminate against women?
Re: Amazon scraps secret AI recruiting tool that showed bias against women
#25The eye opening thing here is not that the AI failed, but why it failed.
At start the AI is like a baby, it doesn't know anything or have any opinions. By teaching it using a set of data, in this case a set of resumes and the outcome then it can form an opinion.
The AI becoming biased tells that the "teacher" was biased also. So actually Amazon's recruiting process seems to be a mess with the technical skills on the resume amounting to zilch, gender and the aggressiveness of the resume's language being the most important (because that's how the human recruiters actually hired people when someone put a resume).
The number of women and men in the data set shouldn't matter (algorithms learn that even if there was 1 woman, if she was hired then it will be positive about future woman candidates). What matters is the rejection rate which it learned from the data.. The hiring process is inherently biased against women.
Technically one could say that the AI was successful because it emulated the current Amazon hiring status.
Re: Amazon scraps secret AI recruiting tool that showed bias against women
#26I hate this industry. Shooting themselves in the foot over and over again because no one can get passed the idea that possibly, women can be just as good at math, logic and computer science - if people would just let them. This never ends. It's just one place after another, when it gets discovered. It never changes.
Why not "women are just not as interested in math, logic and computer science to pursue it AS OFTEN as men"? Why are you not considering this possibility?
They did in the medium past, they didn’t in the recent past, and maybe they will again in the near future.
Re: Amazon scraps secret AI recruiting tool that showed bias against women
#27I hate this industry. Shooting themselves in the foot over and over again because no one can get passed the idea that possibly, women can be just as good at math, logic and computer science - if people would just let them. This never ends. It's just one place after another, when it gets discovered. It never changes.
Why not "women are just not as interested in math, logic and computer science to pursue it AS OFTEN as men"? Why are you not considering this possibility?
Ah, the Damore argument. Besides the fact that his psuedo science has been summarily handled[0], to consider his argument you then have to equally consider the possibility of sexism in academia pressuring women to not study these subjects and societal pressure their whole lives pressuring them to not persue these career paths.
There's also the idea that lack of women scientist "heroes" can be limiting (lack of role models). Basically the idea that if you stack the cards against a population, you're gonna see population-wide effects.
Given these data points, a biased hiring AI contributes to the problem. Therefore, it should be fixed, along with the above points.
[0]https://www.bbc.co.uk/news/world-40865261
[0]https://www.theguardian.com/technology/2017/aug/13/james-dam...
Re: Amazon scraps secret AI recruiting tool that showed bias against women
#28Machine learning should not be used in this way on humans, whether for resume screening or even more dystopic, for sentencing.
Why not?
Seriously: let us take as given that the AI models are biased. Will you also admit that the existing processes are biased? If so, then what we need to ask is which is MORE biased. It might be complaining that we shouldn't release self-driving cars because on rare occasions they cause accidents.
There is, however, another criterion besides how biased it is: how biased it will be in the future. Human-driven processes have the opportunity to become less biased in the future (also the chance to become more biased, but overall things tend to improve). AI processes that are opaque might lock in bias in a fashion that is unreviewable. I believe that the solution is to build AI models that are more transparent -- that could be BETTER (in terms of avoiding bias) than the human-driven processes we use today.
Re: Amazon scraps secret AI recruiting tool that showed bias against women
#29I would guess that the training data for the ML set was the set of all resumes and an indicator of whether the candidate was eventually hired (maybe with supplemental data about how far in the process the candidate got). Could this be a direct indicator of a powerful subconscious bias in Amazon's existing hiring process?
Could this be a direct indicator of a powerful
subconscious bias in Amazon's existing hiring
process?
Imagine a company with 2 men in HR, 2 women in HR, 40 men in engineering, and 10 women in engineering. That's with gender-blind hiring, reflecting only the 4:1 ratio of male to female CS graduates.
If you picked a random male hire, there's a 40/42=95% chance they're an engineer whereas if you picked a random female hire, there's a 10/12=83% chance they're an engineer.
Thus if you look over all hires' CVs, due to Bayes’ law the dataset says being male increases the conditional probability you meet engineering hiring requirements - and the ML system picks up on that.
Re: Amazon scraps secret AI recruiting tool that showed bias against women
#30I hate this industry. Shooting themselves in the foot over and over again because no one can get passed the idea that possibly, women can be just as good at math, logic and computer science - if people would just let them. This never ends. It's just one place after another, when it gets discovered. It never changes.
I understand your frustration, but in my experience recruiting, the primary reason behind there being less women getting hired into engineering roles is almost never raw sexism. Maybe in the 90s, but in the early 10s there was tons of policy around it, bosses were setting the culture, we were doing everything "right." But we were still not hiring that many women, simply because hardly any women ever applied. For chem…
Yes, I am female, I am aware of the statistics. It's frustrating because my life literally gets impacted by automated reasoning such as this. It's not frustrating for anyone who says 'there just aren't enough of you'. That's something that is very easy to say by people who never have to experience that sort of discriminatory practice.
The painful stuff is when it's obvious and provable, because it highlights all the times it can be questionable as to whether it occurs.