Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Algorithmic Monocultures in Hiring

hai.stanford.edu

Post reply on HN
Print view View original

111–120 of 177 posts

Re: Algorithmic Monocultures in Hiring

#111

Post by pc86 »

daft_pink wrote: post #104

Anyone who’s done hiring wouldn’t be shocked by this: We find applicants are more likely to be rejected from every position they apply to than would be predicted by the baseline of each position making statistically independent decisions. Obviously a rejected resume is more likely to be rejected by every other employer and an accepted resume is more likely to be accepted by every other employer. Like online dating, m…

> a rejected resume is more likely to be rejected by every other employer

This makes sense to me, albeit intuitively and in a way I can't articulate.

> an accepted resume is more likely to be accepted by every other employer

but this doesn't necessarily follow from the prior for me. Plenty of people get really good jobs and are really successful in them only after dozens or hundreds of rejections with a nearly-identical resume.

Re: Algorithmic Monocultures in Hiring

#112

Post by paisawalla »

runako wrote: post #70
paisawalla wrote: post #48

Earlier quoted context omitted.

You are selectively adhering to the letter of the law, when the practical effects are already well known and studied. One is not obligated to ignore literature, nor abstain from doing a simple extrapolation from the incentives placed on the table. There is a large body of literature concerning the question "does disparate-impact enforcement cause employers to alter hiring behavior in ways unrelated to actual producti…

> selectively adhering to the letter of the law Are you suggesting that companies should violate the law here? What do you recommend? Edit: charitably, "adhering to the letter of the law" is sometimes shortened to "law-abiding" and is generally what we want.

You've misunderstood the point.

Prior to the beginning of your excerpt is the word "You", meaning the comment's author is the subject, not "companies". I'm saying the commenter is appealing to black letter law for the answer to the question "what happens when..." but we have observational evidence to answer the question.

Re: Algorithmic Monocultures in Hiring

#113

Post by 8note »

wand3r wrote: post #24

Did I miss the part of the article where they break down how they determined race? Is the algorithm blind to race? It looks like they specifically looked at 83k people applying to ~100 companies which notably were Fortune 500 companies. Could there simply be candidate discrepancies here? Hard for me to follow the full methodology but it doesn't necessarily seem either malicious or that well structured. Don't you need…

id expect any algorithm to learn race by other properties in the data?

its going to be in the rest of the data because race has a meaningful correlation, and pleanty of causation with being disadvantaged in real ways, that can also affect the ability to then do certain jobs.

like, the environmental pollution and building interstates and freeways through black communities, on purpose to do bad things to those communities, then results in a bunch of noise and particulate pollution, that is bad for developing brains.

you wont be able to do some meritocratic non-racist hiring without fixing the environmental racism. otherwise youre just mirroring racism other people built for you

Re: Algorithmic Monocultures in Hiring

#114

Post by zeroonetwothree »

efavdb wrote: post #88
verteu wrote: post #22

The paper is here: https://arxiv.org/pdf/2605.27371 They find "disparate impact" of pymetrics across racial groups, but it doesn't seem like they controlled for anything.

They also say that if they do the analysis globally the effect goes away. Curious, does that not imply that if one domain is biased against some group there would be another where the bias was in its favor?

Also has issues of random chance causing these differences. How many different positions are there that have the chance of a 80% effect?

Re: Algorithmic Monocultures in Hiring

#115

Post by zeroonetwothree »

jongjong wrote: post #101

I think the discrimination aspect is downstream from this fact: > We follow 3.4 million people who submit 4 million job applications to 1,700 job postings across 150 employers and 11 industry sectors. Each job application was assessed by an AI hiring tool built by a single third-party vendor. 3.4 million people applying to just 150 employers... Who are all using just 1 platform. WTF. This is where the discrimination…

That’s the platform that gave them the data. I don’t think they claim it’s all the applications of this set of people.

Re: Algorithmic Monocultures in Hiring

#116

Post by avadodin »

xp84 wrote: post #102
59percentmore wrote: post #85

Earlier quoted context omitted.

The assumption is that no one has the authority to decide that all races aren't equally qualified for every position.

"Races" aren't qualified for anything. Neither are star signs or favorite Hogwarts houses. Individuals are qualified or unqualified. If a company happens to end up with less than 1/4 Ravenclaws or not very many Virgos, it doesn't mean hate is a reason. It could be that the Ravenclaws that applied were a bit less qualified than those from the other houses. I guess my point is, doing the statistical analysis for race a…

It could make sense if one was looking to make interventions early on before the candidates reach the selection process.

Don't claim AI is discriminating against non–selects, though.

I doubt companies are using Gr*k to make their hiring decisions.

Re: Algorithmic Monocultures in Hiring

#117

Post by heylook »

pc86 wrote: post #111
daft_pink wrote: post #104

Anyone who’s done hiring wouldn’t be shocked by this: We find applicants are more likely to be rejected from every position they apply to than would be predicted by the baseline of each position making statistically independent decisions. Obviously a rejected resume is more likely to be rejected by every other employer and an accepted resume is more likely to be accepted by every other employer. Like online dating, m…

> a rejected resume is more likely to be rejected by every other employer This makes sense to me, albeit intuitively and in a way I can't articulate. > an accepted resume is more likely to be accepted by every other employer but this doesn't necessarily follow from the prior for me. Plenty of people get really good jobs and are really successful in them only after dozens or hundreds of rejections with a nearly-identi…

The intuition is that they are not truly independent statistical events. Each trial reveals more information about the underlying "quality" of the resume (for passing this trial, not necessarily real world "quality" of the candidate). We are not rolling dice where each toss is fundamentally unrelated to prior tosses.

Re: Algorithmic Monocultures in Hiring

#118

Post by rayiner »

gacgacgac wrote: post #41
wand3r wrote: post #24

Did I miss the part of the article where they break down how they determined race? Is the algorithm blind to race? It looks like they specifically looked at 83k people applying to ~100 companies which notably were Fortune 500 companies. Could there simply be candidate discrepancies here? Hard for me to follow the full methodology but it doesn't necessarily seem either malicious or that well structured. Don't you need…

Yes. You missed it. They are using a test dataset of 83k resumes generated in 2022 for this paper and comparing it as a baseline against their observational data: https://www.nber.org/papers/w29053 The dataset is constructed, deliberately, to hold candidate performance constant and vary the names of candidates to appear to be associated with a specific race.

That’s an earlier paper. This one involves 3 million real applicants, with no control for applicant quality.

Re: Algorithmic Monocultures in Hiring

#119

Post by pc86 »

AStrangeMorrow wrote: post #81
gacgacgac wrote: post #41

Earlier quoted context omitted.

Yes. You missed it. They are using a test dataset of 83k resumes generated in 2022 for this paper and comparing it as a baseline against their observational data: https://www.nber.org/papers/w29053 The dataset is constructed, deliberately, to hold candidate performance constant and vary the names of candidates to appear to be associated with a specific race.

From looking at how that was done, it seems they (the paper you linked) used an older paper which looked at which names are frequent enough and more biased toward a certain demographic (90% of that name occurrence falls within that demographic). But they picked 9 family names per group. Which sounds quite low. And combined that with first names to reach 500 first+last names per group. I wonder how much of the bias we…

Race and socioeconomic status are pretty strongly correlated but I'd imagine it's possible to do a study to see what the extent of each's influence is. You'd need to find "high socioeconomic" names that are also strongly correlated with race(s) themselves correlated with low socioeconomic status and vice versa which honestly might be the hardest part. The disambiguation from a statistical standpoint doesn't seem that difficult once you have the data.

Re: Algorithmic Monocultures in Hiring

#120

Post by pc86 »

anon373839 wrote: post #10
alain94040 wrote: post #4

The European Union passed The Artificial Intelligence Act, which classifies: High-risk – AI applications that are expected to pose significant threats to health, safety, or the fundamental rights of persons. Notably, AI systems used in health, education, recruitment, critical infrastructure management, law enforcement or justice. They are subject to quality, transparency, human oversight and safety obligations That's…

The AI “safety” industry is lobbying for federal preemption so that states won’t have the power to enact these types of sensible regulations.

> > European
Post reply on HN