I tried to do something similar a while ago (for eng hiring specifically). It turned out that the number of grammatical errors and typos mattered way more than anything else on a resume.
That aside, what sucks is that attempts to automate resume scoring rarely look at harder-to-quantify features and focus on low-hanging fruit like keyword occurrences... though in my experience it's such a low-signal document for engineering hiring that the whole thing is a fool's errand.
Hey. I did read the article. It doesn’t support the conclusion OP is drawing. The aim of the AI is to “mechanize the search for talent”. It doesn’t care to, nor have any means to, make decisions “like the recruiters did”. Obviously machines don’t make decisions like humans do. They’re trying to reverse engineer an alternate decisions making process from the previous outcomes.
So the recruiters may or may not have been biased, but if the previous outcomes were (based on the candidate pool) then the AI is sure to have been "taught" that bias. Unless Amazon is willing to accept a) another pool of data or b) that the data will yield bias and apply a correction, the AI is almost guaranteed to be taught the bias.
Yep, I agree a skewed dataset is not good for the task of correcting an unequal distribution and is likely to maintain or even increase it.
The eye opening thing here is not that the AI failed, but why it failed. At start the AI is like a baby, it doesn't know anything or have any opinions. By teaching it using a set of data, in this case a set of resumes and the outcome then it can form an opinion. The AI becoming biased tells that the "teacher" was biased also. So actually Amazon's recruiting process seems to be a mess with the technical skills on the…
This doesn’t seem to be a reasonable conclusion. There is no reason to assume the AI’s assessment methods will mirror those of the recruiters. If Amazon did most of it’s hiring when programming was a task primarily performed by men, and so Amazon didn’t receive many female applicants, they could be unbiased while still amassing a data set that skewed heavily male. The machine would then just correctly assess that fem…
Control question for if you're making a certain intellectual mistake.
The data set will also have skewed heavily against people named "David". Probably only ~1% of the successful applicants.
Would you also expect the machine to be biased against candidates named David?
This will be unpopular but I don't care. What is the evidence that the source data for this 'AI' is biased because the men it came from did not want to hire women? Is there a reserve of unemployed non-male engineers out there? If so what evidence is there of that? Technical talent is both expensive and a rare commodity for tech companies. The non-male engineers I've worked with have always been exceedingly competent,…
Yeah - I'm not even expressing an unpopular opinion, just asking a (leading) question: where are all these women who are chomping at the bit to get into _technical_ positions like programming but find themselves being turned away by biased recruiters? I've never even seen somebody _claim_ that they were a woman who couldn't find a tech job, just people wondering where all the women were.
Last place I worked at had to invest in additional resources to hire several females because the office was mostly male and applicants were overwhelmingly male. We did eventually find a few great female applicants, but it took a lot of work and a lot of time dedicated specifically to that goal.
There does not exist some magical undiscovered pool of talented female engineers that are being turned away by biased recruiters. It's hard enough to find any sort of talented engineers regardless of other factors. Shit, it's not uncommon to recruit from other countries and cover relocation costs these days.
> The number of women and men in the data set shouldn't matter (algorithms learn that even if there was 1 woman, if she was hired then it will be positive about future woman candidates). This is incorrect. The key thing to keep in mind is that they are not just predicting who is a good candidate, they are also ranking by the certainty of their prediction. Lower numbers of female candidates could plausibly lead to low…
> The lower certainty would in turn lead to lower rankings for women even without any bias in the data. I don't think that's true. "No bias" means that gender is irrelevant (i.e. its correlation with outcome is 0%). Therefore the system shouldn't even take it into account - it would evaluate both men and women just by other criteria (experience, technical skills, etc), and it would have equal amounts of data for both…
> "No bias" means that gender is irrelevant
False. If we're talking about the technical statistical definition, bias means systematic deviation from the underlying truth in the data -- see this article by Chris Stucchio with some images for clarification:
"In statistics, a “bias” is defined as a statistical predictor which makes errors that all have the same direction. A separate term — “variance” — is used to describe errors without any particular direction.
It’s important to distinguish bias (making errors with a common direction) from variance which is simply inaccuracy with no particular direction."
> if people would just let them. People do let them. You can't force what people are interested in and you can't let in that which does not exist. In fact many places in tech give preference to women applicants, because they don't apply often and the companies want more women. They're just rare to see. :( There's no grand conspiracy. The truth is much less exciting: Women and men have different preferences, generally…
There's no actual evidence to suggest preference for career type is inherent to being a woman or man. There's plenty showing that women right now have different interests of men, but the cause can easily be societal conditioning - i.e., something that can be fixed.
That probably does play a part but it's a different level than what we're talking about. Its more narrowly about whether people's current interests are being allowed.
Why not "women are just not as interested in math, logic and computer science to pursue it AS OFTEN as men"? Why are you not considering this possibility?
Ah, the Damore argument. Besides the fact that his psuedo science has been summarily handled[0], to consider his argument you then have to equally consider the possibility of sexism in academia pressuring women to not study these subjects and societal pressure their whole lives pressuring them to not persue these career paths. There's also the idea that lack of women scientist "heroes" can be limiting (lack of role m…
"has been summarily handled"
Where does it state the number of applicants, male vs female?
This will be unpopular but I don't care. What is the evidence that the source data for this 'AI' is biased because the men it came from did not want to hire women? Is there a reserve of unemployed non-male engineers out there? If so what evidence is there of that? Technical talent is both expensive and a rare commodity for tech companies. The non-male engineers I've worked with have always been exceedingly competent,…
That is very interesting. What are the stats for unemployed female engineers? Is there a shortage of female representation because female engineers aren’t being hired or is there a shortage because there are actually fewer female engineers? There is a shortage of male therapists and kindergarten teachers: is that because males aren’t being hired or because there are fewer of them in existence?
Also airline pilots, nurses, michelin-starred chefs...
So why is it not ok that women are penalized by being statistically less likely to get an offer yet it’s just fine to penalize men on auto insurance for being statistically more likely to cause an accident?
This will be unpopular but I don't care. What is the evidence that the source data for this 'AI' is biased because the men it came from did not want to hire women? Is there a reserve of unemployed non-male engineers out there? If so what evidence is there of that? Technical talent is both expensive and a rare commodity for tech companies. The non-male engineers I've worked with have always been exceedingly competent,…
I agree with you, but I have to point out, because it's so common: The "this will be unpopular, but I don't care" preface is, I feel, about as damaging to the perception of whatever you're about to say as "I'm not racist, but". To make an effective point, I think you should avoid, as the first thing you say, painting yourself as an underdog brave enough to speak out by preemptively criticising your audience's reactions that haven't even happened yet. That's not to say you should never admit those observations - rather, I would reserve such broad criticism of people's opinions for a separate train of thought or conversation.