Probably being used for age bias as well...
Amazon scraps secret AI recruiting tool that showed bias against women
61–70 of 433 posts
Re: Amazon scraps secret AI recruiting tool that showed bias against women
#62I would guess that the training data for the ML set was the set of all resumes and an indicator of whether the candidate was eventually hired (maybe with supplemental data about how far in the process the candidate got). Could this be a direct indicator of a powerful subconscious bias in Amazon's existing hiring process?
Could this be a direct indicator of a powerful subconscious bias in Amazon's existing hiring process? Maybe - but maybe not. Imagine a company with 2 men in HR, 2 women in HR, 40 men in engineering, and 10 women in engineering. That's with gender-blind hiring, reflecting only the 4:1 ratio of male to female CS graduates. If you picked a random male hire, there's a 40/42=95% chance they're an engineer whereas if you p…
So it depends how you set up the experiment right? I would assume the question you are posing to AI is not how much the resume in question resembles the set of resumes of hired engineers, but rather: given a resume, what is the probability that candidate will eventually be hired?
So the classification function should take into account the resumes of rejected engineers, rather than the pool of resumes of hired employees at Amazon. If someone is seeking a position as an engineer, it is not relevant how much their resume resembles that of HR people, but it is very relevant how much it resembles that of rejected engineering candidates.
If that's the case, then something like having the phrase "women's chess club" in one's resume should not be a meaningful factor for the classifier unless it disproportionately leads to rejection in the current process.
Re: Amazon scraps secret AI recruiting tool that showed bias against women
#63Given that Amazon is so far unable to successfully recommend any product which I actually want, even given the vast dataset of my Amazon purchase history, I am not remotely surprised that their engineers can't successfully develop a people recommendation engine either.
Re: Amazon scraps secret AI recruiting tool that showed bias against women
#64What would happen if you removed gender from all HR and recruiting systems and then retrain the AI? Or for that matter, remove ethnicity, age, creed, etc... Is there any reason we need to be more specific?
Race: Human
Gender: Yes
Age of legal contractual consent: Yes
Re: Amazon scraps secret AI recruiting tool that showed bias against women
#65This is not very surprising - Machine Learning algorithms trained on biased datasets tend to pick up the hidden biases in the training data. It’s important that we be transparent about the training data that we are using, and are looking for hidden biases in it, otherwise we are building biased systems. Fortunately, there are open source tools out there that help audit machine learning models for bias, such as Audit AI, released by pymetrics - https://github.com/pymetrics/audit-ai
Re: Amazon scraps secret AI recruiting tool that showed bias against women
#66Earlier quoted context omitted.
Why not "women are just not as interested in math, logic and computer science to pursue it AS OFTEN as men"? Why are you not considering this possibility?
Ah, the Damore argument. Besides the fact that his psuedo science has been summarily handled[0], to consider his argument you then have to equally consider the possibility of sexism in academia pressuring women to not study these subjects and societal pressure their whole lives pressuring them to not persue these career paths. There's also the idea that lack of women scientist "heroes" can be limiting (lack of role m…
> There's also the idea that lack of women scientist "heroes" can be limiting (lack of role models)
This one is a bit weird, computer guys were always "nerds" and "geeks" to stay away from.
Re: Amazon scraps secret AI recruiting tool that showed bias against women
#67I wish we could move away from resumes for tech role screening anyway, since they convey very little real reliable information. I’ve seen too many great hires from candidates with relatively weak resumes, and failed interviews from candidates with great resumes (and obviously vice versa). I’m not sure what the best alternative should be, though. I am a fan of open source work as a sort of code portfolio, but it doesn…
Unfortunately using open source work, even if only for programming, introduces all sorts of biases as well. A lot of very competent programmers work at jobs that do not have open source contributions and also have families which limit the time they can spend coding after work.
Re: Amazon scraps secret AI recruiting tool that showed bias against women
#68The eye opening thing here is not that the AI failed, but why it failed. At start the AI is like a baby, it doesn't know anything or have any opinions. By teaching it using a set of data, in this case a set of resumes and the outcome then it can form an opinion. The AI becoming biased tells that the "teacher" was biased also. So actually Amazon's recruiting process seems to be a mess with the technical skills on the…
Or maybe it recognised that women were consistently the worst candidates.
Re: Amazon scraps secret AI recruiting tool that showed bias against women
#69Wouldn't an easy way to eliminate bias be to remove any algorithms that use name recognition and gender? If the Ai doesn't have this data to "reason" from wouldn't it level the playing field?
Re: Amazon scraps secret AI recruiting tool that showed bias against women
#70I would guess that the training data for the ML set was the set of all resumes and an indicator of whether the candidate was eventually hired (maybe with supplemental data about how far in the process the candidate got). Could this be a direct indicator of a powerful subconscious bias in Amazon's existing hiring process?
Step into your engineering or computer science department and walk into any upper division class and count the females and count the males. That some companies have upwards of 20% females is more likely indicative of extreme bias in hiring as you're not going to find even remotely close to 20% females there. Enter in most measures of competence and you'll find the division is no different. E.g. - if females were being disproportionately hired because of disproportionately positive performance then this might not be an issue, but there seems to be no evidence for that whatsoever.
This view that the only thing holding people back is some sort of social or systemic bias seems to be based on nothing except ideology. Incidentally, it's an ideology I also used to hold. Like a good egalitarian I pushed my wife away from sociology and into majoring in CS. She did perfectly well, as did I. More than a decade later she works with people and I work with code. I've no regrets there, but it's not so clear that my persuasion was really the best idea.
Norway is another interesting example here. It is considered by many to be the most gender equal location in the world. Yet you'll still find that nurses are primarily female, doctors are primarily male, and all other 'stereotypical' divisions present in most all developed nations. They tried to change these divisions and with extensive effort were able to effect a roughly constant change in some fields. But again, once that push was relinquished things went just about identically to as they were in very short order. Ultimately we're flexible enough that you can manage to fit a square peg into a round hole at times, but once you stop squeezing that peg goes back to what it wants to be.