Live data from Hacker News

A Way to Detect Bias

paulgraham.com

141–150 of 224 posts

Re: A Way to Detect Bias

#141
post #82

The implication of this analysis of http://10years.firstround.com/ is that First Round is biased against founding teams with experience at Amazon, Facebook, Apple, Google, Microsoft or Twitter. Can this be true?

Yes, it can be true. FirstRound could still positively value that experience, but just not be valuing it enough.

Makes sense. Fascinating.

Re: A Way to Detect Bias

#142
Um, the data set pg cites actually shows this to be fallacious.

They excluded Uber from the results. Which, if included, makes the male-run companies look "oversuccessful". What would happen if I excluded the top female-run business, I'd bet that makes the differences between the two groups much smaller.

Given both the small sample size as well as the outsized influence of outliers, drawing conclusions from this population group is going to be fraught with issues.

Re: A Way to Detect Bias

#143

Earlier quoted context omitted.

Compute min(accepted a) and min(accepted B) instead of the means. Dude, your comments are normally smarter than this. Yeah, you can easily fix Grahams's test -- all you need are some numbers that do not exist and that we cannot measure. We're talking about VC's evaluating founders. That does not, and cannot, get reduced to a numerical score. And even if VC's did use some sort of scoring rubric, then we would still no…

A charitable interpretation of what he or she said is this: don't evaluate bias by looking at outcomes of the average applicant, look at the outcomes of the borderline applicants. Even if there is no perfect way to define or measure the minimum acceptable applicant, I think it is reasonable to identify whether applicants were borderline or not.

Isn't that, by the way, what YC has been saying for years in their rejection letters? "We're always surprised by how many of the last companies to make it wind up being the most successful"? Something like that.

Re: A Way to Detect Bias

#144
post #117

Earlier quoted context omitted.

Favoring fair outcome over having fair rules is against everything I believe. We should not strive for a participation trophy culture.

This well-circulated image shows that making everyone a winner has merit in some circumstances. http://static.themetapicture.com/media/funny-equality-justic... The left hand side is fair rules, the right hand side shows a fair outcome

That's alright if the goal is to help the individuals, like welfare. But it's not OK if the goal is to get people to do the most extreme things, like job applicant selection looking for a "best" applicant or baseball team for a best player. A person who can be successful without needing as many boxes to stand on as others.

Don't forget that it's these "best" people who add a vastly disproportionate amount of value to the world. They're the ones who invent new technology and discover new science. We all benefit greatly from their success.

Re: A Way to Detect Bias

#145
This isn't really sound reasoning, for reasons mentioned elsewhere and because of the following.

You need to know that the probability of acceptance is conditionally independent of the "type" of the applicant given the success of the applicant.

For example, consider the following hypothesis for the First Round data: women are more honest than men. A woman presenting a bad idea to a VC will be rejected whereas a man may be able to weasel his way into getting funding. This will make men have a lower success rate, and correspondingly women will have a higher success rate.

However, this isn't really the same thing as having an across-the-board hidden bias against women.

Re: A Way to Detect Bias

#146

Earlier quoted context omitted.

Indeed. The problem is that people frequently infer unfair rules from unequal outcomes, without taking into account the possibility of systematic group differences. Alan: I believe in equality of opportunity, not equality of outcome. Bob: How do you know there isn't equality of opportunity? Alan: Well, just look at how unequal the outcomes are! At this point, Bob would be wise to change the subject, because if he pre…

Apply Occam's Razor to these supposed group differences. Which do you think is a more plausible reality? A. Interviewers prefer candidates who are like themselves, interviewers are mostly white men, therefore most hires are white men. B. The uterus and melanin both inhibit programming ability, interviewers are perfect judges of programming ability, therefore most hires are white men. To look at the present (incomplet…

A straightforward application of evolutionary biology to Homo sapiens yields group differences as the null hypothesis. You've done nothing but construct a ridiculous strawman to refute this. Moreover, discrimination and group differences aren't mutually exclusive—it's possible that Group X's underrepresentation in Field Y is the result of both discrimination and group differences. The only way to know for sure that it's pure discrimination is to show that group differences are negligible. This requires actually measuring them (which in fact has been done in exhausting detail [1]), but even suggesting the possibility of group differences frequently leads to accusations of racism and sexism—as you've just so ably demonstrated.

[1]: See, for example, The Blank Slate by Steven Pinker. Then, once you get over your knee-jerk "That's racist!!!" reflex, take a look—I mean actually read for comprehensionThe Bell Curve by Herrnstein and Murray. Maybe add a little Cavalli-Sforza (via Steve Sailer) to the mix (http://www.vdare.com/articles/052400-cavalli-sforzas-ink-clo...). You can then graduate to basically anything by Arthur Jensen. As a topper, read "Rational"Wiki's entry on Human Biodiversity (http://rationalwiki.org/wiki/Human_biodiversity) and cringe at the smug, supercilious tone, endless strawmanning and distortion, and at the realization that you, too, were once taken in by the ridiculous "mainstream" views. (I certainly was.)

Re: A Way to Detect Bias

#148

On a simple mathematical basis, this is false. Consider two groups of candidates for a scholarship, A and B. We want to select all candidates that have an 80% or better chance of graduation. Group A comes from a population where the chance of graduation is distributed uniformly from 0% to 100% and group B is from one where the chance is distributed uniformly from 10% to 90%, with the same average but less variation i…

> This short comment is not up to pg's usual high standards for his essays.

I can almost here him thinking in response, "if I throw a dog a bone, I don't want to know if it tastes good or not."

Re: A Way to Detect Bias

#149
post #85

Earlier quoted context omitted.

The problem here is language and what our actual objectives are. When people complain about bias, they are not really talking about mathematical bias, but about something else: Their idea of fairness. They are talking about discrimination. And when we are discussing that, we can't really think about whether rules are applied fairly or not, but whether the rules produce the outcomes that we want. Let's go for a ludicr…

Your comment makes sense; indeed, fairness in selection process does not imply that people aren't being discriminated against. For instance, the SAT is fair, but denies those with less opportunities a chance to get into top-tier schools. I can get on board with that. > We can't really think about whether rules are applied fairly or not, but whether the rules produce the outcomes that we want. This is a more explicit…

It's not easy to tell when bias shows up. Collage rankings might look like an unbiased formula, but it's selected so the 'top' schools end up being highly ranked instead of measuring useful things. Things like a high faculty to student ratio don't actually directly have much impact but it's the kind of stat easily gamed by 'top' schools so it's gotten some sort of mythic importance even if these people don't actually teach undergrad classes.

You can find the same inherent bias in many walks of life. Many of the hurtles to becoming a Doctor have nothing to do with being a good Doctor there just there to ensure the right kinds of people get into and out of the program.

Post reply on HN