Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Algorithmic Monocultures in Hiring

hai.stanford.edu

Post reply on HN
Print view View original

101–110 of 177 posts

Re: Algorithmic Monocultures in Hiring

#101

Post by jongjong »

I think the discrimination aspect is downstream from this fact:

> We follow 3.4 million people who submit 4 million job applications to 1,700 job postings across 150 employers and 11 industry sectors. Each job application was assessed by an AI hiring tool built by a single third-party vendor.

3.4 million people applying to just 150 employers... Who are all using just 1 platform. WTF. This is where the discrimination is happening. Why the f do 3.4 million people feel forced to apply to just 150 employers and why the f do all these 150 employers feel forced to use just one platform. WTF.

Re: Algorithmic Monocultures in Hiring

#102

Post by xp84 »

59percentmore wrote: post #85
logicchains wrote: post #28

Earlier quoted context omitted.

>What could justify it? The assumption that applicants from all races are on average equally qualified for every position. Whole subfields of modern academia are based on that assumption.

The assumption is that no one has the authority to decide that all races aren't equally qualified for every position.

"Races" aren't qualified for anything. Neither are star signs or favorite Hogwarts houses.

Individuals are qualified or unqualified. If a company happens to end up with less than 1/4 Ravenclaws or not very many Virgos, it doesn't mean hate is a reason. It could be that the Ravenclaws that applied were a bit less qualified than those from the other houses.

I guess my point is, doing the statistical analysis for race and gender and drawing conclusions, while being completely blind to the one single factor any sane hiring manager should be focusing on -- actual qualifications for the role -- doesn't make any sense.

Re: Algorithmic Monocultures in Hiring

#104

Post by daft_pink »

Anyone who’s done hiring wouldn’t be shocked by this:

We find applicants are more likely to be rejected from every position they apply to than would be predicted by the baseline of each position making statistically independent decisions.

Obviously a rejected resume is more likely to be rejected by every other employer and an accepted resume is more likely to be accepted by every other employer. Like online dating, most employers are looking for some baseline indicators that you are going to be successful and stable.

Re: Algorithmic Monocultures in Hiring

#105

Post by kenjackson »

I think this partially buries the lede: "As a single hiring vendor comes to dominate screening for an industry, it may be more likely that candidates are shut out."

If we move to using just a small number of AI models to help do things like hiring, we will amplify biases and possibly completely lock out portions of the population. We need to be very careful when using AI systems to evaluate people in general -- not because they might be biased (which they might be), but because even a small bias, if used by virtually everyone, can be damning.

Re: Algorithmic Monocultures in Hiring

#106

Post by xp84 »

gacgacgac wrote: post #41
wand3r wrote: post #24

Did I miss the part of the article where they break down how they determined race? Is the algorithm blind to race? It looks like they specifically looked at 83k people applying to ~100 companies which notably were Fortune 500 companies. Could there simply be candidate discrepancies here? Hard for me to follow the full methodology but it doesn't necessarily seem either malicious or that well structured. Don't you need…

Yes. You missed it. They are using a test dataset of 83k resumes generated in 2022 for this paper and comparing it as a baseline against their observational data: https://www.nber.org/papers/w29053 The dataset is constructed, deliberately, to hold candidate performance constant and vary the names of candidates to appear to be associated with a specific race.

Wow. So, all the 'people' and 'resumes' involved are fake, but they submitted them to real jobs?

Cool.

In any event, I'd happily support a ban on all parts of the ATS that could be involved in automated approval, rejection, or scoring being able to see candidate names. But I sense the author of this has a bigger agenda.

Re: Algorithmic Monocultures in Hiring

#107

Post by zeroonetwothree »

daft_pink wrote: post #104

Anyone who’s done hiring wouldn’t be shocked by this: We find applicants are more likely to be rejected from every position they apply to than would be predicted by the baseline of each position making statistically independent decisions. Obviously a rejected resume is more likely to be rejected by every other employer and an accepted resume is more likely to be accepted by every other employer. Like online dating, m…

Yes I don’t understand why this is surprising or problematic at all?

Actually the fact that they found this result didn’t hold in a different dataset is especially weird.

Re: Algorithmic Monocultures in Hiring

#109

Post by pc86 »

kenjackson wrote: post #105

I think this partially buries the lede: "As a single hiring vendor comes to dominate screening for an industry, it may be more likely that candidates are shut out." If we move to using just a small number of AI models to help do things like hiring, we will amplify biases and possibly completely lock out portions of the population. We need to be very careful when using AI systems to evaluate people in general -- not b…

If you want to make meaningful change in this avenue you really can't use words like "bias" or "systemic" because anywhere from 49-51% of the population will immediately shut down upon hearing that. Someone can argue (and many do to varying levels of success) that systemic bias doesn't exist, which means this doesn't exist, which means there no problem.

However, "this AI model can decide that some subset of people, perhaps random, perhaps not, are simply not hirable for any job" makes sense to most people regardless of political bent.

Re: Algorithmic Monocultures in Hiring

#110

Post by paisawalla »

SiempreViernes wrote: post #75
paisawalla wrote: post #48

Earlier quoted context omitted.

You are selectively adhering to the letter of the law, when the practical effects are already well known and studied. One is not obligated to ignore literature, nor abstain from doing a simple extrapolation from the incentives placed on the table. There is a large body of literature concerning the question "does disparate-impact enforcement cause employers to alter hiring behavior in ways unrelated to actual producti…

That's not particularly surprising nor objectionable, of course legislation that reminds employers they shouldn't discriminate based on race changes practice even for companies that aren't actually caught doing it. To act like it's bad that people of colour have a more fair chance of getting employed because of some piece of legislation is simply insidious. It's just been over a month since black people lost the righ…

I don't start from the conclusion that disparities are evidence of racism.
Post reply on HN