Live data from Hacker News

Amazon scraps secret AI recruiting tool that showed bias against women

reuters.com

221–230 of 433 posts

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#221

Anyone who has worked at tech companies and has been involved in the hiring process knows the following: The best thing to be right now is a woman engineer. You can easily get hired within the week. Unfortunately this doesn't seem to be well known outside of those involved in hiring.

Is this anecdotal? Are you insinuating that women have an upper hand now for reasons beyond their skillset?

Yes. There is a bonus to being a woman in this situation, all other things being equal.

Silicon Valley did a good bit on this https://www.youtube.com/watch?v=Dek5HtNdIHY

It's funny because it's true.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#222

Earlier quoted context omitted.

Yeah - I'm not even expressing an unpopular opinion, just asking a (leading) question: where are all these women who are chomping at the bit to get into _technical_ positions like programming but find themselves being turned away by biased recruiters? I've never even seen somebody _claim_ that they were a woman who couldn't find a tech job, just people wondering where all the women were.

It's taking a little longer than it should, but people are finally starting to realize that the actual reason there aren't as many women in software is because they've chosen not to be. They're wired differently and therefore have different interests.

I don't think that's quite true, I think this is more of a nurture vs nature thing.

My father got me into programming, and my colleagues who are women also have a backstory with someone supporting their interest in development.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#223
post #103

I wish we could move away from resumes for tech role screening anyway, since they convey very little real reliable information. I’ve seen too many great hires from candidates with relatively weak resumes, and failed interviews from candidates with great resumes (and obviously vice versa). I’m not sure what the best alternative should be, though. I am a fan of open source work as a sort of code portfolio, but it doesn…

> author-blind reviews Should I ever be in a position to hire a colleague, I wouldn't ever do so without having a chat with them. I spend 8hrs a day in an office with my colleagues (sometimes more than with my wife & kid) and the ones I can't stand is about the only thing wrong with my job. If we can't even see the person's face over some gender bias hysteria then I wonder how the hell we got here. People should just…

>I spend 8hrs a day in an office with my colleagues (sometimes more than with my wife & kid) and the ones I can't stand is about the only thing wrong with my job.

There's plenty of people I would never ever spend time with outside of work, and try to minimize my time with at work.

But that's fine, because 'cthalupa would like to have a beer with you after work' isn't part of the requisites for doing a job on my team. The 'Finding people that fit in with the culture'/'Finding people that I don't mind being around' is how you get monocultures and a lack of diversity in your team.

>If we can't even see the person's face over some gender bias hysteria then I wonder how the hell we got here.

Gender bias is a real thing, a big deal, and certainly not hysteria. There's a lot of ways to reduce it. It doesn't necessarily require never seeing someone's face - though I think automating "skills" related interviews could be a good thing - because you can start with having a structured interview program where you have specific questions to ask and a specific rubric to grade against. Making sure you have solid, unbiased questions, and measure the answers evenly against the same rubric solves the majority of the problem.

>People should just get over the fact that men and women are different.

Well, of course men and women are different. But even for jobs that involve heavy labor, this isn't actually significant - while the average woman has less physical strength than the average man, the type of woman who applies for that sort of job has self-selected into it, and is almost certainly more capable of doing that sort of work than the average woman - meaning it's still not a good indicator even for areas where the differences are largest.

For a white collar job, like the type we're discussing? It's even less relevant. There are extremely few times where you should ever care about gender when it comes to hiring.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#224
post #25

The eye opening thing here is not that the AI failed, but why it failed. At start the AI is like a baby, it doesn't know anything or have any opinions. By teaching it using a set of data, in this case a set of resumes and the outcome then it can form an opinion. The AI becoming biased tells that the "teacher" was biased also. So actually Amazon's recruiting process seems to be a mess with the technical skills on the…

The article didn't specify how they labeled resumes for training. You're assuming that it was based on whether or not the candidate was hire. Nobody with an iota of experience in machine learning would do something like that. (For obvious reasons: you can't tell from your data whether people you did not hire were truly bad.)

A far more reasonable way would be to take resumes of people who were hired and train the model based on their performance. For example, you could rate resumes of people who promptly quit or got fired as less attractive than resumes of people who stayed with the company for a long time. You could also factor in performance reviews.

It is entirely possible that such model would search for people who aren't usually preferred. E.g. if your recruiters are biased against Ph.D.'s, but you have some Ph.D.'s and they're highly productive, the algorithm could pick this up and rate Ph.D. resumes higher.

Now, you still wouldn't know anything about people whom you didn't hire. This means there is some possibility your employees are not representative of general population and your model would be biased because of that.

Let's say your recruiters are biased against Ph.D.'s and so they undergo extra scrutiny. You only hire candidates with a doctoral degree if they are amazing. This means within your company a doctoral degree is a good predictor of success, but in the world at large it could be a bad criteria to use.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#225
post #25

The eye opening thing here is not that the AI failed, but why it failed. At start the AI is like a baby, it doesn't know anything or have any opinions. By teaching it using a set of data, in this case a set of resumes and the outcome then it can form an opinion. The AI becoming biased tells that the "teacher" was biased also. So actually Amazon's recruiting process seems to be a mess with the technical skills on the…

> The number of women and men in the data set shouldn't matter (algorithms learn that even if there was 1 woman, if she was hired then it will be positive about future woman candidates). This is incorrect. The key thing to keep in mind is that they are not just predicting who is a good candidate, they are also ranking by the certainty of their prediction. Lower numbers of female candidates could plausibly lead to low…

It's wrong even if their model doesn't output a certainty (not all classifiers do). Almost all ML algorithms optimize the expected classification error under the training distribution. So if the training data contains 90% men, it's better to classify those men at 100% accuracy and women at 0% accuracy, than it is to classify both with 89.9% accuracy. Any unsophisticated model will do this.

gp: "The number of women and men in the data set shouldn't matter (algorithms learn that even if there was 1 woman, if she was hired then it will be positive about future woman candidates)."

This is false for typical models.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#226

Earlier quoted context omitted.

It's taking a little longer than it should, but people are finally starting to realize that the actual reason there aren't as many women in software is because they've chosen not to be. They're wired differently and therefore have different interests.

I see this argument a lot but your exposition is less inflammatory than most. Please note this: when you tell a mixed-gender group of people that all women are "wired" to not like programming, you are telling the women who do like programming (which, let's be real, is most of them on Hacker News) that they're in effect not real women (or defective women, in an engineering sense). Or that they don't count. Since you'r…

>you are telling the women who do like programming (which, let's be real, is most of them on Hacker News) that they're in effect not real women

woah what? no that is not at all what he's saying. If someone tells me "most american men like the NFL" and I don't like the NFL, I would be insane to take that as someone telling me "you're not a real man" and think they're trying to "tell me what a real man is." I can see how someone who is perpetually trying to be a victim might take such a hardline stance, though.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#227
post #11

I hate this industry. Shooting themselves in the foot over and over again because no one can get passed the idea that possibly, women can be just as good at math, logic and computer science - if people would just let them. This never ends. It's just one place after another, when it gets discovered. It never changes.

Why not "women are just not as interested in math, logic and computer science to pursue it AS OFTEN as men"? Why are you not considering this possibility?

I haven't seen anything suggesting women are less interested, but there is some support for the idea that before college girls who are interested in math., etc., are more likely than boys to have other subjects that interest them more.

There was a study published a while back that looked at PISA data and found that girls and boys were pretty evenly represented among the kids who were at the top in STEM [1].

But it also found that for the boys in that group quite often STEM was the only thing they were outstanding at. In other areas they were average to good.

For the girls, on the other hand, they were often excellent at something else in addition to STEM, with them often even being better at that something else than they were in STEM.

People have a tendency to pursue a career in one of the areas they are very good at.

This suggests that boys who are very good in math, etc., are more likely than similarly good girls to pursue it as a career because that is their only choice if they want to go into something they are very good at. The girls are more likely to have math, etc., as one of two or more possible careers in areas they are very good at.

In pop culture terms, STEM boys are more like Martin Prince, and STEM girls are more like Lisa Simpson.

[1] I didn't save the link and have failed to find it with Google. Anyone have it?

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#228
post #206

Earlier quoted context omitted.

It's taking a little longer than it should, but people are finally starting to realize that the actual reason there aren't as many women in software is because they've chosen not to be. They're wired differently and therefore have different interests.

But there are fewer women in software now than there were 30 years ago. Are women today "wired differently" from their mothers? Single-generation changes in behavior aren't genetic. They're social.

But you're not comparing the same thing generation to generation. The job of developer has changed massively, the number of developers, the expectations, the salary. Sociaety has also changed, and not just in culture but in income distribution, etc etc.

If tomorrow we say that you have to do 30 chinups to be a waitress, and the job will involve regular fistfights then we count the number of waitresses by gender and say "it must be cultural", we're kind of missing the point. Or if we say "OK now waitresses make 200k and are respected" and watch the numbers shift.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#229

Earlier quoted context omitted.

I see this argument a lot but your exposition is less inflammatory than most. Please note this: when you tell a mixed-gender group of people that all women are "wired" to not like programming, you are telling the women who do like programming (which, let's be real, is most of them on Hacker News) that they're in effect not real women (or defective women, in an engineering sense). Or that they don't count. Since you'r…

"Since you're presumably not a woman, and they are, they object to your seeming to be taking it upon yourself to tell them what a woman is." That idea is complete poison. In a debate where you're both completely uninformed, the anecdotal evidence of experience is relevant. In any discussion where reason, numbers, research are involved, the gender/race of the person making the argument is irrelevant to the argument. T…

I'm sorry, I don't follow how your argument (which is valid) is a response to mine. I said people tend to object to being imposed an identity by people who do not share it ("all women hate programming", "all Yankees eat too much"). (And to a lesser extent they also object to the same thing by ingroup members, but are less comfortable expressing it. If my parent was in fact a woman, her viewpoint would most likely still be pretty unpopular.)

Arguments being inflammatory does not make them invalid. It is fine to have conversations that include inflammatory arguments, if they are made politely, which parent did. But ignoring that the argument is inflammatory, and/or that the group the argument refers to are in fact intelligent people involved in the discussion that may have themselves an opinion, is lacking in empathy. Rhetoric is founded in empathy; that is why it is an art, and not merely a technique.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#230
post #199

Earlier quoted context omitted.

Last place I worked at had to invest in additional resources to hire several females because the office was mostly male and applicants were overwhelmingly male. We did eventually find a few great female applicants, but it took a lot of work and a lot of time dedicated specifically to that goal. There does not exist some magical undiscovered pool of talented female engineers that are being turned away by biased recrui…

Just being female is a qualification at your workplace?

a rather curious question... what prompted it?
Post reply on HN