Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Algorithmic Monocultures in Hiring

hai.stanford.edu

Post reply on HN
Print view View original

41–50 of 177 posts

Re: Algorithmic Monocultures in Hiring

#41

Post by gacgacgac »

wand3r wrote: post #24

Did I miss the part of the article where they break down how they determined race? Is the algorithm blind to race? It looks like they specifically looked at 83k people applying to ~100 companies which notably were Fortune 500 companies. Could there simply be candidate discrepancies here? Hard for me to follow the full methodology but it doesn't necessarily seem either malicious or that well structured. Don't you need…

Yes. You missed it. They are using a test dataset of 83k resumes generated in 2022 for this paper and comparing it as a baseline against their observational data: https://www.nber.org/papers/w29053

The dataset is constructed, deliberately, to hold candidate performance constant and vary the names of candidates to appear to be associated with a specific race.

Re: Algorithmic Monocultures in Hiring

#42

Post by redsocksfan45 »

moate wrote: post #35
bakugo wrote: post #9

> To put this in perspective: If the AI had recommended Black and Asian candidates at the same rate as it recommended the most-favored group (typically white applicants) Some people just can't help but put their biases on display at every opportunity, even when it comes to the most minute details.

Where do you think this sentence shows bias? The phrase "most-favored" means, "most recommended by the AI relative to the field". What did you think this sentence meant?

[dead]

Re: Algorithmic Monocultures in Hiring

#44

Post by jazz9k »

We can't take blanket percentages as a reason for racial bias. Were they all equally qualified?

Too many of these studies only focus on percentages and the end result is unqualified candidates getting hired from minority groups at the expense of qualified ones.

Re: Algorithmic Monocultures in Hiring

#46

Post by ortusdux »

Ayres, I., Banaji, M. and Jolls, C. (2015), Race effects on eBay. The RAND Journal of Economics, 46: 891-917. https://doi.org/10.1111/1756-2171.12115

"Cards held by African-American sellers sold for approximately 20% ($0.90) less than cards held by Caucasian sellers, and the race effect was more pronounced in sales of minority player cards."

Re: Algorithmic Monocultures in Hiring

#47

Post by gacgacgac »

ApolloFortyNine wrote: post #38

I truly don't doubt it's possible for the AI to be 'racist'. >If the AI had recommended Black and Asian candidates at the same rate as it recommended the most-favored group (typically white applicants), 40,000 more of their applications would have advanced to the next stage of hiring. I don't think this is the right benchmark here, or at least, it would be very interesting if the actual outcome, offer or rejected, wa…

You are misreading this sentence. This sentence is saying: "Using a constructed dataset of resumes, whose only difference was a name change, we would anticipate a system evaluating on qualifications to produce an equal distribution of candidates across names. Our observed result was highly unequal, and that warrants further investigation."

Re: Algorithmic Monocultures in Hiring

#48

Post by paisawalla »

gacgacgac wrote: post #33
paisawalla wrote: post #30

Earlier quoted context omitted.

This is an application of the disparate impact doctrine. Even facially neutral policies are considered suspect if they produce results that correlate against protected groups, irrespective of intent. This doctrine is the basis for much of employment law. It is a significant reason why employers don't administer IQ tests (or equivalents) to screen candidates since ~the 90s. A common objection to the doctrine is that i…

Importantly, the rule is not used to resolve racial discrimination claims. It's purely meant as the first test to evaluate whether a deeper dive is warranted. Fast, first pass data analysis tools are very useful for spotting unintended consequences.

You are selectively adhering to the letter of the law, when the practical effects are already well known and studied. One is not obligated to ignore literature, nor abstain from doing a simple extrapolation from the incentives placed on the table.

There is a large body of literature concerning the question "does disparate-impact enforcement cause employers to alter hiring behavior in ways unrelated to actual productivity or discrimination?" and the answer is largely "yes". As you suggested elsewhere in this discussion, Google may be useful.

Re: Algorithmic Monocultures in Hiring

#49

Post by gacgacgac »

jazz9k wrote: post #44

We can't take blanket percentages as a reason for racial bias. Were they all equally qualified? Too many of these studies only focus on percentages and the end result is unqualified candidates getting hired from minority groups at the expense of qualified ones.

Please read the study or at least the comments here before jumping to the conclusion. Yes, they used constructed resumes, so the qualifications were exactly the same. And no, literally no one is suggesting this proves racial discrimination. It's applying the four fifths rule, a fast, coarse evaluation that is used to identify if maybe theres worth investigating more for a conclusive evidence of racial discrimination.

The authors are saying it's worth doing more research, because in a controlled data set the results appear unbalanced.

Re: Algorithmic Monocultures in Hiring

#50

Post by foolserrandboy »

logicchains wrote: post #15

Could the AI actually see the race of the applicants? Or was it just discriminating on the basis of some factor it found that was correlated with race, like SAT scores?

It rejected Asians more because of their higher SAT scores? If it’s not directly based on applicants disclosing their ethnicity then probably something more obvious like names.
Post reply on HN