Earlier quoted context omitted.
> Obviously a rejected resume is more likely to be rejected by every other employer and an accepted resume is more likely to be accepted by every other employer. But that wasn't the case for non-algorithmic screening. From the paper: "By contrast, we find that when first round screening is not mediated by a single screening procedure, systemic rejections are close to the baseline. To support the empirical validity of…
The paper with the non-algorithmic screening used synthetic resumes. I rather suspect that they didn't generate a realistic distribution of qualifications levels.
Algorithmic Monocultures in Hiring
141–150 of 177 posts
Re: Algorithmic Monocultures in Hiring
#142The European Union passed The Artificial Intelligence Act, which classifies: High-risk – AI applications that are expected to pose significant threats to health, safety, or the fundamental rights of persons. Notably, AI systems used in health, education, recruitment, critical infrastructure management, law enforcement or justice. They are subject to quality, transparency, human oversight and safety obligations That's…
There's no reason to single out AI vs any other approach to the same topics.
Re: Algorithmic Monocultures in Hiring
#143Earlier quoted context omitted.
This is an application of the disparate impact doctrine. Even facially neutral policies are considered suspect if they produce results that correlate against protected groups, irrespective of intent. This doctrine is the basis for much of employment law. It is a significant reason why employers don't administer IQ tests (or equivalents) to screen candidates since ~the 90s. A common objection to the doctrine is that i…
And a common rebuttal to the objection is that systemic racism is often difficult to untangle in a way that produces a neat chain of cause and effect (not least of which because discrimination can happen unconsciously or secretly); because the impact exists whether intent can be shown or not, the desire remains to ameliorate that impact. If the issue happens upstream of the defendant to a claim - generally an organiz…
We have a "disparate impact" and nobody can prove what proportion of it is due to things like parental income or childhood education as opposed to racism on the part of the employer. Because the former considerations are real contributors, the metric can regularly be expected to exceed the threshold even if the contribution of racism by the employer was zero. Doesn't that imply that we're essentially accusing people of racism at random?
> because the impact exists whether intent can be shown or not, the desire remains to ameliorate that impact.
The median household income for Asian Americans of Indian ethnicity is more than double those of Burmese ethnicity:
https://en.wikipedia.org/wiki/List_of_ethnic_groups_in_the_U...
This is objectively a disparate impact and likely shows up in several other metrics in addition to income. Disparate results can almost universally be obtained by arbitrarily segmenting the population into different groups and comparing the midpoints. Americans of Australian ancestry have a higher median income than those of Irish ancestry, Bolivians higher than Cubans. The result is often because the lower down group has a history of being oppressed.
What reasoned means can we use to determine which groups get the benefit of these methods to ameliorate the disparity and which don't? What should be done about the inherent impossibility of doing them simultaneously, e.g. because hiring a South African woman over a Haitian man would reduce the disparity on one axis while increasing it on another? Notice that considering each group separately could result in unconditional liability because either available alternative puts you over the threshold for one group or the other.
> If the issue happens upstream of the defendant to a claim - generally an organization being sued by an individual with fewer resources - it incentivizes such entities to push for changes upstream, so that they don't get stuck with the bill.
Do we want to apply this logic to other things? The median income in California and New York are significantly higher than they are in Alabama or West Virginia and they have higher ranked public schools. We can correspondingly expect that when applicants from different states apply for the same job, the ones from California and New York (even if they're the same race etc.) are more likely to be selected because they had more advantages growing up, even though none of them chose where they were born.
By the same reasoning we should then have the federal government penalize employers for hiring the applicants from the more affluent states so that it "incentivizes such entities to push for changes upstream, so that they don't get stuck with the bill." Does it make sense to do that?
Re: Algorithmic Monocultures in Hiring
#144Earlier quoted context omitted.
You are reading a paper without understanding the language of the paper. Adverse Impact has a specific meaning, and in this case it's specifically meaning that Black candidates were selected only four fifths as often as white candidates when their qualifications were identical. The study is only suggesting that further investigation is warranted.
Thanks. It was unclear reading the article and linked paper. I didn't follow the citation/link trail far enough.
Re: Algorithmic Monocultures in Hiring
#145Many people seem to think racism begins and ends with using a slur. You can usually get a measure of this by seeing someone's reaction to the statement: > There is no such thing as anti-white racism. If you find yourself wanting to disagree with that then, I'm sorry but you simply don't know what racism is. Racism is pervasive, insidious and systemic. A good example in the hiring space is what's called the "second sy…
> > There is no such thing as anti-white racism. > If you find yourself wanting to disagree with that then, I'm sorry but you simply don't know what racism is. You are saying that if you think anti-white racism can exist, you don't know what racism is. That's obviously ludicrous.
The colloquially version, which means "prejudice based on race" and a second version, which specific groups and people have advocated for, which means something like "structural oppression through cultural and governmental means". It's more complicated than just that, but it's a fairly narrow term for them.
So when one person says "there's no such thing as anti-white racism", you hear, "No one's prejudiced against white people for being white!" Obviously, that's ludicrous.
But that person is likely using the, I have no idea what to call it, "advocate definition" maybe, definition would which preclude anti-white racism from existing within that narrow definition of racism.
So it's a debate where people aren't speaking the same version of a language, convinced each other are uninformed, reactionary or stupid.
Re: Algorithmic Monocultures in Hiring
#146I think this partially buries the lede: "As a single hiring vendor comes to dominate screening for an industry, it may be more likely that candidates are shut out." If we move to using just a small number of AI models to help do things like hiring, we will amplify biases and possibly completely lock out portions of the population. We need to be very careful when using AI systems to evaluate people in general -- not b…
But LLMs are statistical models. They are aggregating all biases into a general super bias. And they're all converging towards the same solutions.
Re: Algorithmic Monocultures in Hiring
#147We can't take blanket percentages as a reason for racial bias. Were they all equally qualified? Too many of these studies only focus on percentages and the end result is unqualified candidates getting hired from minority groups at the expense of qualified ones.
Please read the study or at least the comments here before jumping to the conclusion. Yes, they used constructed resumes, so the qualifications were exactly the same. And no, literally no one is suggesting this proves racial discrimination. It's applying the four fifths rule, a fast, coarse evaluation that is used to identify if maybe theres worth investigating more for a conclusive evidence of racial discrimination.…
Re: Algorithmic Monocultures in Hiring
#148I then went on to work for multiple firms that placed a premium on candidates from Ivy League/Top Tier (Stanford/Duke etc) candidates.
This taught me that:
- Their are pros and cons to any selection criteria.
- There are smart people everywhere. One of the smartest people I ever worked for spent several years in prison for drug dealing. He was on par with many of the Managing Directors I've worked for
- There was a study where they asked big bank recruiters which school consistently produced people who were excellent employees 2-3 years out from hiring and the answer was Penn State (not my alma mater)
- There used to be "manager's choice" hires where managers had 1 slot in a training program where they could select whoever they wanted. Sometimes that was terrible. Sometimes that person was top of their training program.
- Smart people are just as capable as creating problems as less intelligent people. Smart people, in some ways, are better at creating problems. Especially if the incentives reward them for creating those problems.
Re: Algorithmic Monocultures in Hiring
#149I think this partially buries the lede: "As a single hiring vendor comes to dominate screening for an industry, it may be more likely that candidates are shut out." If we move to using just a small number of AI models to help do things like hiring, we will amplify biases and possibly completely lock out portions of the population. We need to be very careful when using AI systems to evaluate people in general -- not b…
It's strange to say it might be biased. Bias is absolutely impossible to avoid, especially with how today's "AI" works. You might be able to avoid it with a panel of AI, similar to how we try to avoid it by using panels of humans, but even that turns out to be contentious and not surefire. I have feeling with AI it'll be even worse, since folks / companies can pass the buck (similar to how health insurance companies…
Unless you're taking the "there are multiple mathematically incompatible ways to define bias" view of the topic, just do what's already known best practice for high-bureaucracy human review. Which is too define an overly-pedantic standard rubric.
Re: Algorithmic Monocultures in Hiring
#150Earlier quoted context omitted.
What evidence would disprove the claim that systemic racism is the cause of a persistent disparity?
Why is this the one time someone is expected to disprove a claim rather than the claimant being expected to provide evidence? If you're making the claim you need to provide the evidence. Most people would say that a persistent disparity means it's possible there is discrimination, but it's not definitive proof.