A Way to Detect Bias

What's "exceptional"? I've known many women in my career and many of them were mediocre. Some were in positions of authority and some weren't. Another commenter lists several "mediocre" women in the corporate and political world (and leaves out some big ones, like Meg Whitman). If you're talking about becoming a CEO, you have to be an "exceptional" man to get there too, in absolute terms.

I feel like the root erroneous assumption here is that an equal amount of people of all types are interested in the same things and that the only reason any group becomes more represented than another is that the others are getting alienated or funneled out somewhere along the way. That is a completely incorrect and invalid assumption. The fact that there are a lot more non-English-speakers in janitorial work in the US (when I worked as a janitor, I was 1 of 2 English speakers on the 12-person janitorial staff) doesn't necessarily mean the janitorial manager is biased against English speakers; it means that due to external considerations, like the fact that almost all other jobs require you to speak the native language, non-English-speakers are better suited for janitorial work, and therefore people do the logical thing, apply for work that they can do, and end up comprising a larger section of the application pool.

People make decisions based on social, cultural, and physical expectations of them, and there's not anything wrong with that. By and large, women do not have an interest in computer sciencey or entrepreneurial work. It's OK if a woman does, but it's also OK to note that most women don't. There's nothing we need to fix about it. Most women don't want to do it, and there's no reason to force them.

Why do you see fewer women becoming CEOs? Because fewer women want that kind of job and fewer women are qualified for that kind of job due to the biological realities of humanity that require women to take time out for pregnancy and child-rearing (sorry denialists, I didn't invent biology and choose that only women could bear and nurse children, so don't take it up with me), and the social and cultural expectations that have formed around these biological realities. In short, the serious applicant pool includes only a very small amount of women, so only a very small number of women obtain that position.

People, most of whom clearly are not that good at math, are being really harsh on Paul Graham.

Graham is mostly right, but slightly incorrect. In particular, suppose group A has the distribution f(x) and B has the distribution g(x).

If f(x) and g(x) are shaped significantly differently past the cutoff, then mean(H(x-C)f(x)) and mean(H(x-c)g(x)) might not agree even though there is no bias by construction. (Here H(x) is a step function and C the cutoff).

However, there is an easy fix: compute the minima of the support of the distribution rather than mean. min(H(x-C)f(x)) = min(H(x-C)g(x)) = C.

In practice, measure the weakest male and weakest female to be accepted in your sample set, or some similar approximation.

I'm pretty sure this is a valid frequentist hypothesis test. I've got half a proof worked out on paper already. It depends very weakly (and non-parametrically) on f(x) and g(x), but it works in basically the exact way Graham wants it to. Every counterexample I can think of is really pathological. My next blog post will probably be a proof of this.

All this negativity is really an overreaction. I know it's fun to totally debunk someone on details, but these are mostly fixable details.

This is only a bias if the stereotype effect is stronger on measurements than in real life. If stereotype threat affects test performance and real performance the same way, then it means that the stereotyped group is truly inferior.

Do you have evidence that stereotype threat hurts test performance more than real performance?

(Of course, in a hypothetical world which only eliminated the stereotype, the group would cease to be inferior. I.e., the inferiority is based on context, and is not intrinsic.)

Wouldn't the weakest male and weakest female founders be underperforming more due to individual factors than bias?

Maybe a mean of the lower quartile would remove some the the noise.

This presupposes that we should not accept unequal representation among groups. I don't believe that, and I don't think it's an implicit Western value. People should be allowed to flourish according to their natural advantages. The value is to try not to make an early judgment and exclude people based on assumptions about their group's capability, whether that exclusion is based on the group's perceived disadvantage or advantage. The idea is that the individual shouldn't be held accountable for things he had nothing to do with, and shouldn't be assumed to be automatically compliant with stereotypes. I don't see a need to start throttling groups that are doing "too well".