Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Way to Detect Bias

paulgraham.com

Post reply on HN
Print view View original

151–160 of 224 posts

Re: A Way to Detect Bias

#151

Post by devalier »

tedsanders wrote: post #131
devalier wrote: post #126

Earlier quoted context omitted.

Compute min(accepted a) and min(accepted B) instead of the means. Dude, your comments are normally smarter than this. Yeah, you can easily fix Grahams's test -- all you need are some numbers that do not exist and that we cannot measure. We're talking about VC's evaluating founders. That does not, and cannot, get reduced to a numerical score. And even if VC's did use some sort of scoring rubric, then we would still no…

A charitable interpretation of what he or she said is this: don't evaluate bias by looking at outcomes of the average applicant, look at the outcomes of the borderline applicants. Even if there is no perfect way to define or measure the minimum acceptable applicant, I think it is reasonable to identify whether applicants were borderline or not.

A charitable interpretation of what he or she said is this: don't evaluate bias by looking at outcomes of the average applicant, look at the outcomes of the borderline applicants.

That is fine, that is what he was saying. The point is that his solution is completely impractical for the original goal of finding an objective, statistically valid way of measuring whether bias exists. "Borderline" cannot be measured objectively, only by subjective rubric scoring. And when you only measure the borderline candidates, you have reduced an already way-to-small sample even further.

Re: A Way to Detect Bias

#152

Post by hugh4 »

gizmo wrote: post #7

A related observation (which I've been making for a long time) is that the absence of mediocre women in positions of power is strong evidence of bias. Men can succeed when they're mediocre, but women have to be exceptional. Likewise for minorities.

I've seen plenty of mediocre women and minorities in positions of power.

For a bipartisan example, consider Barack Obama and Sarah Palin.

Re: A Way to Detect Bias

#153

Post by lexcorvus »

baddox wrote: post #132
rewqfdsa wrote: post #114

Earlier quoted context omitted.

Right, and unwillingness to consider the possibility of group differences comes from a quasi-religious devotion to the blank slate model of human nature. The way radical egalitarians see it, we're not only equal in dignity, but in potential. That's a pretty view, but it's inconsistent with reality, and radical egalitarians need to come up with increasingly implausible explanations to explain everyday circumstances th…

There is a simple explanation for differences in abilities between groups that has nothing to do with their genetics are so-called natural ability: the fact that groups often grow up around other members of their group. Both nature and nurture are largely in common for many groups, so it could easily be either that causes the observed differences in ability.

could easily be either

Or both. They're not mutually exclusive. Do Jamaican sprinters excel because they grow up around other sprinters or because they are blessed with natural ability? Yes. Simply put, or != xor.

Re: A Way to Detect Bias

#154

Post by forrestthewoods »

hibikir wrote: post #85
WildUtah wrote: post #20

On a simple mathematical basis, this is false. Consider two groups of candidates for a scholarship, A and B. We want to select all candidates that have an 80% or better chance of graduation. Group A comes from a population where the chance of graduation is distributed uniformly from 0% to 100% and group B is from one where the chance is distributed uniformly from 10% to 90%, with the same average but less variation i…

The problem here is language and what our actual objectives are. When people complain about bias, they are not really talking about mathematical bias, but about something else: Their idea of fairness. They are talking about discrimination. And when we are discussing that, we can't really think about whether rules are applied fairly or not, but whether the rules produce the outcomes that we want. Let's go for a ludicr…

But the stated objective of the root article was failed. It concluded that First Round Capital is biased against females. There is insufficient data to support that claim! It might actually be that FRC is biased FOR females. But the process leading up to FRC was so biased against females that females are still at a net loss.

Re: A Way to Detect Bias

#155

Post by lexcorvus »

rewqfdsa wrote: post #114
lexcorvus wrote: post #112

Earlier quoted context omitted.

Indeed. The problem is that people frequently infer unfair rules from unequal outcomes, without taking into account the possibility of systematic group differences. Alan: I believe in equality of opportunity, not equality of outcome. Bob: How do you know there isn't equality of opportunity? Alan: Well, just look at how unequal the outcomes are! At this point, Bob would be wise to change the subject, because if he pre…

Right, and unwillingness to consider the possibility of group differences comes from a quasi-religious devotion to the blank slate model of human nature. The way radical egalitarians see it, we're not only equal in dignity, but in potential. That's a pretty view, but it's inconsistent with reality, and radical egalitarians need to come up with increasingly implausible explanations to explain everyday circumstances th…

Hey rewqfdsa, maybe shoot me an email some time. Address is in profile.

Re: A Way to Detect Bias

#156

Post by MarkMc »

pg wrote: post #91
WildUtah wrote: post #20

On a simple mathematical basis, this is false. Consider two groups of candidates for a scholarship, A and B. We want to select all candidates that have an 80% or better chance of graduation. Group A comes from a population where the chance of graduation is distributed uniformly from 0% to 100% and group B is from one where the chance is distributed uniformly from 10% to 90%, with the same average but less variation i…

It's true that this test assumes groups of applicants are roughly equal in (distribution of) ability. That is the default assumption in most conversations I've been involved in about bias, and particularly the example I used, but I'll add something making that explicit.

A small request: When you amend an article after it is published please note the change in a footnote. It took me a long time to realise the top comment on HN was referring to an older version of the article that didn't mention the equal distribution of ability.

Re: A Way to Detect Bias

#157

Post by Natsu »

jonahx wrote: post #139
hibikir wrote: post #85

Earlier quoted context omitted.

The problem here is language and what our actual objectives are. When people complain about bias, they are not really talking about mathematical bias, but about something else: Their idea of fairness. They are talking about discrimination. And when we are discussing that, we can't really think about whether rules are applied fairly or not, but whether the rules produce the outcomes that we want. Let's go for a ludicr…

This is a smart insight, although in fairness the article suggests, in its female founder example, that there was discrimination against women -- that given men and women of equal ability, men were more likely to be chosen. Condemning that inequality is different from affirming that selection should be altered to produce the outcomes people view as fair. One is saying, "Don't discriminate against Xs." The other is sa…

I think the grandparent example is pointing out that it will be hard to use this for sexism claims precisely because studies have shown that the variance in ability in male and female populations is, in fact, different. The studies I'm aware of show more men at both extremes of the bell curve. So more men at the very top and bottom in IQ measurements[1].

There's a genetic basis for this, as well: women have two copies of each chromosome, whereas men have X and Y, so there's no second copy to take over in men, leading to more extreme outcomes, whether good or bad.

Now of course we ought to treat every group of people fairly, but we do need to examine our priors when doing so, especially when proposing ways to detect and punish people who may be thinking bad things, consciously or otherwise.

[1] We may not know just what 'IQ' is, but we do know that tests of mental ability all correlate with each other, suggesting an underlying factor. This, in turn, can be correlated with many other things, like success (or lack thereof).

Re: A Way to Detect Bias

#158

Post by gozo »

rewqfdsa wrote: post #107
55555 wrote: post #93

Earlier quoted context omitted.

Great comment. There are two types of fairness, (a) fair rules, and (b) fair outcome.

Favoring fair outcome over having fair rules is against everything I believe. We should not strive for a participation trophy culture.

"Favoring fair outcome over having fair rules"

Which essentially no one does? People favor fair outcome because they don't think the rules are, or can be, fair and often as a proxy for rules becoming more fair.

Re: A Way to Detect Bias

#159

Post by vii »

yummyfajitas wrote: post #49
WildUtah wrote: post #20

On a simple mathematical basis, this is false. Consider two groups of candidates for a scholarship, A and B. We want to select all candidates that have an 80% or better chance of graduation. Group A comes from a population where the chance of graduation is distributed uniformly from 0% to 100% and group B is from one where the chance is distributed uniformly from 10% to 90%, with the same average but less variation i…

Graham's intuition is assuming equality of the two distributions. As I noted in a different comment here, you can pretty easily fix Graham's test. Compute min(accepted a) and min(accepted B) instead of the means. In your example, the min of the accepted distributions would both work out to be 80%.

This assumes that the populations of A and B are of the same size. A larger sample will tend to have a lower minimum under many real world distributions - a sample of one will have its minimum equal to its maximum.

Another reason the use of mins here is not helpful, is that adding one equally awful accepted candidate to group A and B would then remove whatever bias there was according to the test, which is not what we want the test to indicate.

The idea by PG is a rough rule of thumb and breaks down trivially - suppose VC fund X were to accept all candidates, but group A was worse than B, the test would falsely imply that the fund was biased.

It's unfortunate the idea was dressed up in statistical persiflage because it isn't rigorous -- it's a rough guideline. To make it rigorous wold be very hard: either the abilities of the candidate populations would have to be measured very closely (unrealistic), or a more scientific experiment conducted (A-B test where candidates from each group are included or excluded opposite to the prior decision, which would need big groups).

Re: A Way to Detect Bias

#160

Post by anecon2 »

For a formalized and empirical version of this argument applied to the entirety of the US economy, check out the following article: The Allocation of Talent and U.S. Economic Growth by Hsieh et al. (http://klenow.com/HHJK.pdf). It quantifies the gains from the decreases in misallocation of women and african americans as racial discrimination in employment decreased over the past 50 years.
Post reply on HN