Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amazon scraps secret AI recruiting tool that showed bias against women

reuters.com

Post reply on HN
Print view View original

301–310 of 433 posts

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#301

Post by jahewson »

macinjosh wrote: post #93

This will be unpopular but I don't care. What is the evidence that the source data for this 'AI' is biased because the men it came from did not want to hire women? Is there a reserve of unemployed non-male engineers out there? If so what evidence is there of that? Technical talent is both expensive and a rare commodity for tech companies. The non-male engineers I've worked with have always been exceedingly competent,…

Some reasons why I disagree with this polemic:

1. There's no reason to expect that these women will be unemployed - they just won't be working for Amazon. That's all we know. No point going looking for them.

2. You can't assign intent to hiring decisions made in the training data - there's no reason to believe that men (and why single them out?) "did not want to hire women". Maybe they did. Maybe they have no idea that they're biased - maybe the women making such hiring decisions are just as biased. We have no idea.

3. The evidence that the AI is biased, is that.... the AI is biased. Which means that the training data is biased. Why that is, is a great question - it may reflect unconscious bias in the hiring process, or more obvious old-fashioned biases. It may reflect that the model amplifies some minor bias in the training data and turns it into something much bigger. We don't know.

So yeah, it's biased - the question is why.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#302

Post by sol_remmy »

beat wrote: post #291
sol_remmy wrote: post #279

Earlier quoted context omitted.

Possibility 1: Male-dominated fields discriminate against females. Possibility 2: Those fields are male-dominated because they can't get into female-dominated fields. Men do not work as teachers because the media has painted men as "sex crazed". Most mothers would be uncomfortable with having a male 4th grade teacher for their daughter. Many women would be uncomfortable having a male gynecologist or a male nurse help…

You have a number of issues with your narrative. "Quit the profession or go part time in order to raise kids". So what other reasons do women have for quitting the profession, other than because men are too victimized to be stay at home dads? -- edit: fwiw, I googled stats. According to the American Association of Medical Colleges, 2017 was the first year ever that female medical school enrollment was greater than ma…

> Men do not work as teachers because the media has painted men as "sex crazed". Most mothers would be uncomfortable with having a male 4th grade teacher for their daughter.

> Many women would be uncomfortable having a male gynecologist or a male nurse helping them deliver their baby.

And what is your opinion of the above bit of my previous post (since you avoided that in your answer?)

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#303

Post by UncleMeat »

exo762 wrote: post #11
trustmath wrote: post #2

I hate this industry. Shooting themselves in the foot over and over again because no one can get passed the idea that possibly, women can be just as good at math, logic and computer science - if people would just let them. This never ends. It's just one place after another, when it gets discovered. It never changes.

Why not "women are just not as interested in math, logic and computer science to pursue it AS OFTEN as men"? Why are you not considering this possibility?

We did consider this possibility. Then we did research and the experts and they found that this isn't supported by evidence.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#304

Post by pcmaffey »

beat wrote: post #198

Garbage in, garbage out. The system failed because they were trying to solve the wrong problem, or maybe more specifically, didn't solve the problem that led to the problems with the AI. Amazon was treating the hiring problem as an efficiency problem alone, and ignoring the bias problem. So they wound up training the AI to do a shitty job much faster than humans ever could be shitty - and, by analyzing the data in a…

> So they wound up training the AI to do a shitty job much faster than humans ever could be shitty

If nothing else, modeling our existing behavior in this way is a great use case for ML. As it allows us to "fast forward" and thus--hopefully!--identify our flaws based on modeled iterations.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#305

Post by titzer »

screye wrote: post #292
lalaland1125 wrote: post #40

Earlier quoted context omitted.

> The number of women and men in the data set shouldn't matter (algorithms learn that even if there was 1 woman, if she was hired then it will be positive about future woman candidates). This is incorrect. The key thing to keep in mind is that they are not just predicting who is a good candidate, they are also ranking by the certainty of their prediction. Lower numbers of female candidates could plausibly lead to low…

> The lower certainty would in turn lead to lower rankings for women even without any bias in the data. This is not true. Probabilistic-ly speaking, if we are computing P(hiring | gender); Lower certainty means there is a high variance in prior over women. However, over a large dataset, the "score" would almost certainly be equal to the mean of the distribution, and be independent of the variance. In simpler words, i…

There's not enough information about how their ML algorithm works, nor how large their dataset was for any of the above reasoning to be justified. Fwiw, many ranking functions do indeed take certainty into account, penalizing populations with few data points.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#306

Post by komali2 »

mike22223333 wrote: post #261
komali2 wrote: post #114

Earlier quoted context omitted.

There's no actual evidence to suggest preference for career type is inherent to being a woman or man. There's plenty showing that women right now have different interests of men, but the cause can easily be societal conditioning - i.e., something that can be fixed.

Sweden tried it. It failed.

sigh

I want to consider your, er, argument, in best possible faith, but you've given me almost nothing to work with here.

Sweden tried what?

Failure means what?

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#307

Post by zakm »

maym86 wrote: post #245
lawnchair_larry wrote: post #129

So why is it not ok that women are penalized by being statistically less likely to get an offer yet it’s just fine to penalize men on auto insurance for being statistically more likely to cause an accident?

Because men are statistically more likely to cause auto accidents. There is no evidence that women are not statistically more likely to be bad at getting good work done at companies like Amazon. It's pretty simple.

I'm sure that's true, but do you have any evidence for it?

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#308

Post by beat »

sol_remmy wrote: post #302
beat wrote: post #291

Earlier quoted context omitted.

You have a number of issues with your narrative. "Quit the profession or go part time in order to raise kids". So what other reasons do women have for quitting the profession, other than because men are too victimized to be stay at home dads? -- edit: fwiw, I googled stats. According to the American Association of Medical Colleges, 2017 was the first year ever that female medical school enrollment was greater than ma…

> Men do not work as teachers because the media has painted men as "sex crazed". Most mothers would be uncomfortable with having a male 4th grade teacher for their daughter. > Many women would be uncomfortable having a male gynecologist or a male nurse helping them deliver their baby. And what is your opinion of the above bit of my previous post (since you avoided that in your answer?)

My opinion is it's not worth deigning to answer.

Re: Amazon scraps secret AI recruiting tool that showed bias against women

#310

Post by beat »

naasking wrote: post #300
beat wrote: post #298

Earlier quoted context omitted.

Why are you assuming I have no idea of the data available? Because I question the default narrative? I do find it interesting and noteworthy that gender disparities have grown in STEM while shrinking in other fields. But I believe my explanation accounts for that - that STEM has become more prestigious, which draws men, which forces out women. The "well known psychological attitude" is begging the question, which see…

> Why are you assuming I have no idea of the data available? Because I question the default narrative? Because you're throwing out wild, unsupported speculation to salvage your narrative, and the original post of yours to which I replied had at least 4 elementary factual errors. > But I believe my explanation accounts for that - that STEM has become more prestigious, which draws men, which forces out women. That's no…

Sexism kept women out of medicine and law for centuries. It's only very recently that this has changed. Women were not even admitted to Harvard Law School until 1950.

I do think there's a universal component, though, as sexism is seen across virtually all cultures.

Post reply on HN