Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Algorithmic Monocultures in Hiring

hai.stanford.edu

Post reply on HN
Print view View original

161–170 of 177 posts

Re: Algorithmic Monocultures in Hiring

#161

Post by paisawalla »

runako wrote: post #127
paisawalla wrote: post #112

Earlier quoted context omitted.

You've misunderstood the point. Prior to the beginning of your excerpt is the word "You", meaning the comment's author is the subject, not "companies". I'm saying the commenter is appealing to black letter law for the answer to the question "what happens when..." but we have observational evidence to answer the question.

> we have observational evidence to answer the question. Isn't the point that the observational evidence amounts to the companies in question steer clear of illegal behavior? There are anti-money laundering laws, so banks institute procedures to help them comply. Yes, we expect companies to change their processes so they don't break the law. That's the point of the law. I am confused with what you think companies sho…

The thread you're weighing in on is about whether the four-fifths rule is merely diagnostic, or functionally punitive.

You're arguing about something else, using the form "given we know that racism is happening, and furthermore we know where and how it is happening, why shouldn't we just do these Totally Cost Free and Obviously Good Things that are just like fighting money laundering"

Well, I just don't think any of these things are as evident as you seem to assume they are. Also fwiw I grew up in the US, where I was called all sorts of slurs -- like, the real ones you probably censor in your internal monologue when you see them written -- throughout K-12 education. I still don't believe in the existence of pervasive and oppressive racism the way you seem to assume it.

Re: Algorithmic Monocultures in Hiring

#163

Post by pc86 »

pc86 wrote: post #119
AStrangeMorrow wrote: post #81

Earlier quoted context omitted.

From looking at how that was done, it seems they (the paper you linked) used an older paper which looked at which names are frequent enough and more biased toward a certain demographic (90% of that name occurrence falls within that demographic). But they picked 9 family names per group. Which sounds quite low. And combined that with first names to reach 500 first+last names per group. I wonder how much of the bias we…

Race and socioeconomic status are pretty strongly correlated but I'd imagine it's possible to do a study to see what the extent of each's influence is. You'd need to find "high socioeconomic" names that are also strongly correlated with race(s) themselves correlated with low socioeconomic status and vice versa which honestly might be the hardest part. The disambiguation from a statistical standpoint doesn't seem that…

Another hindrance to collecting data in the first place is that idiots will read this and downvote it because they think statistics are racist, I guess?

Re: Algorithmic Monocultures in Hiring

#164

Post by pc86 »

alexpotato wrote: post #148

I went to a state school. I then went on to work for multiple firms that placed a premium on candidates from Ivy League/Top Tier (Stanford/Duke etc) candidates. This taught me that: - Their are pros and cons to any selection criteria. - There are smart people everywhere. One of the smartest people I ever worked for spent several years in prison for drug dealing. He was on par with many of the Managing Directors I've…

> There used to be "manager's choice" hires where managers had 1 slot in a training program where they could select whoever they wanted. Sometimes that was terrible. Sometimes that person was top of their training program.

This seems like a great idea to me if you institute a feedback loop so managers who pick trash eventually lost the option to pick, and managers who pick rock stars eventually get more picks / more responsibility.

Re: Algorithmic Monocultures in Hiring

#165

Post by pc86 »

slashdave wrote: post #135
kenjackson wrote: post #105

I think this partially buries the lede: "As a single hiring vendor comes to dominate screening for an industry, it may be more likely that candidates are shut out." If we move to using just a small number of AI models to help do things like hiring, we will amplify biases and possibly completely lock out portions of the population. We need to be very careful when using AI systems to evaluate people in general -- not b…

It is also illegal

What law does it violate? It's not even clear to me what "it" you're referring to.

Re: Algorithmic Monocultures in Hiring

#166

Post by sdellis »

adammarples wrote: post #139
sdellis wrote: post #80

Earlier quoted context omitted.

Unless you believe that Black people are racially inferior, I think this is simply evidence of racial discrimination at a systemic level, from education through employment. AI merely reenforces the systems built to favor white people.

There are many other potential explanatory factors than your simple binary. Black people in America started in a very bad and difficult position, only a few generations ago, with huge racial discrimination, no money, and generational distrust of institutions. That is a factor that will affect what you see today without any current system of racial discrimination or inferiority.

If I hear you correctly, the lack of reparations toward Black people in America is more to blame for the discrepancy than systemic racism? Perhaps it could be both?

I am getting downvoted because it's hard to admit that AI only reinforces the culture that it is trained on. It is the perfect technology to keep systemic racism in place, all while being the perfect scapegoat for lack of personal or corporate accountability.

Re: Algorithmic Monocultures in Hiring

#168

Post by [deleted] »

paisawalla wrote: post #92
59percentmore wrote: post #82

Earlier quoted context omitted.

And a common rebuttal to the objection is that systemic racism is often difficult to untangle in a way that produces a neat chain of cause and effect (not least of which because discrimination can happen unconsciously or secretly); because the impact exists whether intent can be shown or not, the desire remains to ameliorate that impact. If the issue happens upstream of the defendant to a claim - generally an organiz…

What evidence would disprove the claim that systemic racism is the cause of a persistent disparity?

[deleted]

Re: Algorithmic Monocultures in Hiring

#169

Post by drillsteps5 »

daft_pink wrote: post #104

Anyone who’s done hiring wouldn’t be shocked by this: We find applicants are more likely to be rejected from every position they apply to than would be predicted by the baseline of each position making statistically independent decisions. Obviously a rejected resume is more likely to be rejected by every other employer and an accepted resume is more likely to be accepted by every other employer. Like online dating, m…

This does not sound right to me. It would be correct if all companies and their hiring managers had the same requirements/looking for candidates with the same qualifications - but they're not.

Obviously as a hiring manager you're looking for a hard working individual with a number of successfully completed projects and glowing referrals from multiple places of employment, but you're also looking for a person with expertise in particular technologies/industries/whatever other areas of expertise. To a large extent requirements for each role are unique, however many do have some overlap.

So being rejected from one position might simply mean there's a misalignment between what the company is looking for and what the individual has. Which might not be the case with other companies.

So if we're seeing increasing number of candidates being consistently rejected at multiple places the question "why" is a valid one.

Re: Algorithmic Monocultures in Hiring

#170

Post by kcexn »

diob wrote: post #132
kenjackson wrote: post #105

I think this partially buries the lede: "As a single hiring vendor comes to dominate screening for an industry, it may be more likely that candidates are shut out." If we move to using just a small number of AI models to help do things like hiring, we will amplify biases and possibly completely lock out portions of the population. We need to be very careful when using AI systems to evaluate people in general -- not b…

It's strange to say it might be biased. Bias is absolutely impossible to avoid, especially with how today's "AI" works. You might be able to avoid it with a panel of AI, similar to how we try to avoid it by using panels of humans, but even that turns out to be contentious and not surefire. I have feeling with AI it'll be even worse, since folks / companies can pass the buck (similar to how health insurance companies…

Everyone knows there is bias. The problem this article highlights is that by delegating screening and human judgment to a few AI vendors those vendors will bias all employers in the same way.
Post reply on HN