Amazon scraps secret AI recruiting tool that showed bias against women • Hacker News Board

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#291

Post by beat » Wed, Oct 10, 2018, 7:47 PM UTC

sol_remmy wrote: post #279

beat wrote: post #272

Earlier quoted context omitted.

Possibility 1: Female-dominated fields discriminate against men. Possibility 2: Those fields are female-dominated because they can't get into male-dominated fields. So what do the pay and prestige look like for female fields, vs male fields? Well, take medical. Nurses (low prestige, low pay) are >90% female. Doctors (high prestige, high pay) are about 70% male. This suggests to me that there's indeed a huge level of…

Possibility 1: Male-dominated fields discriminate against females. Possibility 2: Those fields are male-dominated because they can't get into female-dominated fields. Men do not work as teachers because the media has painted men as "sex crazed". Most mothers would be uncomfortable with having a male 4th grade teacher for their daughter. Many women would be uncomfortable having a male gynecologist or a male nurse help…

You have a number of issues with your narrative. "Quit the profession or go part time in order to raise kids". So what other reasons do women have for quitting the profession, other than because men are too victimized to be stay at home dads?

-- edit: fwiw, I googled stats. According to the American Association of Medical Colleges, 2017 was the first year ever that female medical school enrollment was greater than male medical school enrollment. I also went to graduation by year as far back as 2002, and it has always been more men than women. So yeah, your statistics are bullshit. Care to offer a source? --

And mind you, being a stay at home parent is considered a low-prestige, low-pay role. To the extent that it's discouraged for men, that's a result of a sexism that puts men in a dominant role and demeans them for doing "women's work".

The idea that men aren't teachers because the media paints them as sex-crazed is absurd. The gender disproportion of teachers existed long before the media mentioned such things at all. And you offer no evidence whatsoever for the assertion.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#292

Post by screye » Wed, Oct 10, 2018, 7:52 PM UTC

lalaland1125 wrote: post #40

fuscy wrote: post #25

The eye opening thing here is not that the AI failed, but why it failed. At start the AI is like a baby, it doesn't know anything or have any opinions. By teaching it using a set of data, in this case a set of resumes and the outcome then it can form an opinion. The AI becoming biased tells that the "teacher" was biased also. So actually Amazon's recruiting process seems to be a mess with the technical skills on the…

> The number of women and men in the data set shouldn't matter (algorithms learn that even if there was 1 woman, if she was hired then it will be positive about future woman candidates). This is incorrect. The key thing to keep in mind is that they are not just predicting who is a good candidate, they are also ranking by the certainty of their prediction. Lower numbers of female candidates could plausibly lead to low…

> The lower certainty would in turn lead to lower rankings for women even without any bias in the data.

This is not true.

Probabilistic-ly speaking, if we are computing P(hiring | gender); Lower certainty means there is a high variance in prior over women. However, over a large dataset, the "score" would almost certainly be equal to the mean of the distribution, and be independent of the variance.

In simpler words, if there was a frequency diagram of scores for each gender (most likely bell curves), then only the peak of the bell curve would matter. The flatness / thinness of the curve would be completely irrelevant to the final score. The peak is the mean, and the flatness is the uncertainty. Only the mean matters.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#293

Post by HashHishBang » Wed, Oct 10, 2018, 7:53 PM UTC

user812 wrote: post #274

HashHishBang wrote: post #117

Hold on here. This article seems to have buried a pretty important piece of information wayyy down in the middle of the text. > Gender bias was not the only issue. Problems with the data that underpinned the models’ judgments meant that unqualified candidates were often recommended for all manner of jobs, the people said. With the technology returning results almost at random, Amazon shut down the project, they said.…

The media is no longer reporting things. You can't make money with reporting. The media is actively creating narratives, and one of the narratives that people are fed nowadays is that women are victims. Men and women are pitted against each other. Due to the way the media has evolved people consume their own biases and most often just read the headlines.

I mean, yeah you're not wrong. I try not to be too cynical about the whole thing even if I think the narrative is suspect. Yes women and minority representation in tech is a potential issue but I really want to know more about the AI recommendation system for potential hires. Especially if it was giving out spurious recommendations.

It's amazon, I can't imagine how many millions went into something like that. We'll almost certainly not get a postmortem but it's definitely intriguing.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#294

Post by ramblerouser » Wed, Oct 10, 2018, 7:54 PM UTC

dx87 wrote: post #271

paavoova wrote: post #257

Earlier quoted context omitted.

The parent post? Evidently, for whatever reason, male applicants and coworkers are a problem for their organization, so they go out of their way and invest in "additional resources" to hire female candidates from the hiring pool. They're not hiring on merit, but on gender. How exactly do you hire for female candidates without discriminating against the "overwhelming" body of male applicants? I would be really interes…

If it's anything like conferences trying to increase diversity numbers, they don't specifically target one gender, they focus their recruiting efforts on sites and locations that typically have a much higher representation, like a girls college, or a girls hacker group. That way they aren't actively saying "we won't accept male applicants", but they ensure that 90% of the applicants will be from the target demographi…

>We did eventually find a few great female applicants, but it took a lot of work and a lot of time dedicated specifically to that goal.

How is this not sexism? Replace female with male in the above quote and tell me it isnt sexist.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#295

Post by darkerside » Wed, Oct 10, 2018, 7:55 PM UTC

beat wrote: post #272

sol_remmy wrote: post #256

Earlier quoted context omitted.

> And that's where hands come up empty. Maybe anti-male sexism prevalent in the health care and education fields is causing women to prefer those fields. Fix the sexism in health care/education. Elementary teachers should be 50% men. Nurses should be 50% men. Instead those fields are 90%(!) women! That is a HUGE level of bias and discrimination

Possibility 1: Female-dominated fields discriminate against men. Possibility 2: Those fields are female-dominated because they can't get into male-dominated fields. So what do the pay and prestige look like for female fields, vs male fields? Well, take medical. Nurses (low prestige, low pay) are >90% female. Doctors (high prestige, high pay) are about 70% male. This suggests to me that there's indeed a huge level of…

The way you're ranking occupations has an implicit bias. Let's rank them for work/life balance. Nurses are busy and work long hours, but when the work day is over, they go home until the next shift. Doctors go home, and possibly get paged to come right back.

Is it possible men and women weight values differently when selecting occupations?

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#296

Post by rhcom2 » Wed, Oct 10, 2018, 7:59 PM UTC

sol_remmy wrote: post #212

beat wrote: post #207

Earlier quoted context omitted.

It doesn't need to be deliberate or conscious. It can be incidental. Or maybe women just aren't as smart as men.

No body thinks that. You're way outside the overton window. Why is the health care field heavily biased in favor or female nurses and doctors? Are women smarter than men when it comes to biology/anatomy?

I don't believe there is a heavy bias in favor of female doctors, I believe it is a field still majority male, although becoming almost equal. Nurses were traditionally the only actual healthcare profession open to women so it makes sense they would be overrepresented there.

Male nurses now actually can find they have an advantage in hiring because they often have an easier time with the lifting and physical labor being a nurse often requires.

My point is the comparison between nursing and programming is not strong.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#297

Post by danShumway » Wed, Oct 10, 2018, 8:05 PM UTC

kareemsabri wrote: post #45

fuscy wrote: post #25

The eye opening thing here is not that the AI failed, but why it failed. At start the AI is like a baby, it doesn't know anything or have any opinions. By teaching it using a set of data, in this case a set of resumes and the outcome then it can form an opinion. The AI becoming biased tells that the "teacher" was biased also. So actually Amazon's recruiting process seems to be a mess with the technical skills on the…

This doesn’t seem to be a reasonable conclusion. There is no reason to assume the AI’s assessment methods will mirror those of the recruiters. If Amazon did most of it’s hiring when programming was a task primarily performed by men, and so Amazon didn’t receive many female applicants, they could be unbiased while still amassing a data set that skewed heavily male. The machine would then just correctly assess that fem…

This is a subtle point but worth stating -- AI does not mirror or copy human reasoning.

AI is designed to get the same results as a human. How it gets to those results is often very, very different. I'm having trouble finding it, but there was an article a while back trying to do focus tracking between humans and computers for image recognition. What they found was that even when computers were relatively consistent with humans in results, they often focused on different parts of the image and relied on different correlations.

That doesn't mean that Amazon isn't biased. I mean, let's be honest, it probably is; there's no way a company this large is going to be able to perfectly filter or train every employee and on average tech bias trends against women. BUT, the point is that even if Amazon were to completely eliminate bias from every single hiring decision it used in its training data, an AI still might introduce a racial or gendered bias on its own if the data were skewed or had an unseen correlation that researchers didn't intend.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#298

Post by beat » Wed, Oct 10, 2018, 8:06 PM UTC

naasking wrote: post #285

beat wrote: post #267

Earlier quoted context omitted.

Okay, so TWO generations. Big deal. It still dismisses the "born that way" nonsense argument. "'programming' as a profession used to be regarded as an offshoot of secretarial work, which was dominated by women". Which begs the question of why women dominated secretarial work (and still do), while as programming became a more respected and better paying profession, it became male-dominated.

> Okay, so TWO generations. Big deal. It still dismisses the "born that way" nonsense argument. It really doesn't. Unless you seriously think punch card programming is the same as modern programming, or that the fact that only women were secretaries that did programming somehow provides data on the relative strengths and inclinations of women and men for programming work at that time. Look, it's clear that you have n…

Why are you assuming I have no idea of the data available? Because I question the default narrative?

I do find it interesting and noteworthy that gender disparities have grown in STEM while shrinking in other fields. But I believe my explanation accounts for that - that STEM has become more prestigious, which draws men, which forces out women.

The "well known psychological attitude" is begging the question, which seems par for the course on responses here. Is this psychological attitude biological, or social? And if it's biological, how do we explain significant changes in professional proportions that have happened over a mere one or two generations? It seems like a very poor explanation for what you're asserting, contradicting your own stated facts.

If it's social, however, we're back to my explanation - as the prestige of formerly female-dominated careers rises, they become more attractive to men, to the point where men dominate them. It's a much simpler explanation, with no contradictions.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#299

Post by cimmanom » Wed, Oct 10, 2018, 8:11 PM UTC

androidgirl wrote: post #222

Dirlewanger wrote: post #202

Earlier quoted context omitted.

It's taking a little longer than it should, but people are finally starting to realize that the actual reason there aren't as many women in software is because they've chosen not to be. They're wired differently and therefore have different interests.

I don't think that's quite true, I think this is more of a nurture vs nature thing. My father got me into programming, and my colleagues who are women also have a backstory with someone supporting their interest in development.

Yup. Someone telling us that programming was something for us - not just being exposed to media and advertising and social pressures that continually suggest (and did even more so in the 80's and 90's) that computing was for socially inept males.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#300

Post by naasking » Wed, Oct 10, 2018, 8:20 PM UTC

beat wrote: post #298

naasking wrote: post #285

Earlier quoted context omitted.

> Okay, so TWO generations. Big deal. It still dismisses the "born that way" nonsense argument. It really doesn't. Unless you seriously think punch card programming is the same as modern programming, or that the fact that only women were secretaries that did programming somehow provides data on the relative strengths and inclinations of women and men for programming work at that time. Look, it's clear that you have n…

Why are you assuming I have no idea of the data available? Because I question the default narrative? I do find it interesting and noteworthy that gender disparities have grown in STEM while shrinking in other fields. But I believe my explanation accounts for that - that STEM has become more prestigious, which draws men, which forces out women. The "well known psychological attitude" is begging the question, which see…

> Why are you assuming I have no idea of the data available? Because I question the default narrative?

Because you're throwing out wild, unsupported speculation to salvage your narrative, and the original post of yours to which I replied had at least 4 elementary factual errors.

> But I believe my explanation accounts for that - that STEM has become more prestigious, which draws men, which forces out women.

That's not an explanation at all. Why would prestige drive away women? Just because there are men there? Or you think men drawn to prestige don't want women around? Or you think men just flood into any field that has some form of prestige thus drowning out women? So then why aren't the careers they left suddenly dominated by women because all the men left for more prestige? And where are all these men coming from since we have rough equal numbers of men and women? Why are janitors and dangerous jobs dominated by men since those aren't prestigious?

The fact that you think this explains anything or is free of contradictions is frankly bizarre, and just reinforces my point that if you're really interested in this field, you need to more read more and speculate less.

> The "well known psychological attitude" is begging the question, which seems par for the course on responses here. Is this psychological attitude biological, or social?

Likely both, since there's plenty of evidence of things vs. people in toddlers, and this innate preference no doubt gets reinforced and magnified.

In the end, your scoffing at the original poster and "subtly" implying that he's sexist for a remark that is actually well grounded in facts is exactly the problem with debating people on this subject.

Yes, there is sexism in STEM, just like there is in most other fields, but sexism didn't keep women out of medicine or law, they just pushed through and staked their claim. The fact that women haven't done this for STEM which is far less of an old boys' club already suggests something else is at play, and the fact that the same trends are seen across disparate cultures already suggests strongly there's a universal component.