A Way to Detect Bias

This in my eyes amounts to a simplification that is neat mathematically, but removes most of the useful information. I admit though that if the minimums did not match very well, that is an obvious sign of bias.

Edit: I suppose what I'm essentially saying is, what if minimums do match, then we cannot rule out bias.

When people complain about bias, they are not really talking about mathematical bias, but about something else: Their idea of fairness. They are talking about discrimination. And when we are discussing that, we can't really think about whether rules are applied fairly or not, but whether the rules produce the outcomes that we want.

Let's go for a ludicrous example: We'll accept all applicants whose IQ is higher than their weight in pounds. We'll be explicitly discriminating against heavy people, but at the same time, we have pretty clear implicit biases against men, and ethnic groups who tend to be taller. We might as well have said that we prefer children and Japanese women. There's no need for mathematical bias: The bias comes from the rule selection.

So, in your example, if our actual objective is to graduate an even amount of people from groups A and B, we have to, explicitly, make it easier for group B to get the scholarship. And many times organizations have objectives like that.

As a more real example, let's consider a police department. If the objective is to have a racial makeup that represents the community, and different races have different drop-out rates, the candidate selection will prefer one kind over the other, precisely to counter the drop-out differential.

So when regular people, and not mathematicians, discuss bias, the mathematical definition is unimportant. The one important thing is our stated objectives.

The issue, interestingly, is that there are a lot of women (or biased-against group) between the median performance and 160% performance who are being rejected.

In other words, only the best women can get accepted, which means that above-median (but not superstars) are getting rejected, while many more above-median men are getting accepted.

This has the ring of truth to me (as pg says, it's the definition of bias).

Assumption: There is no fundamental difference between a female and a male founder for achieving start-up success (average rates and variance/distribution of rates is the same)

Observation: VC funded start-ups with female founders are (on average) 60% more successful than start-ups with male founders

Hypothesis: VC funding is biased against female founders. The ones that do receive funding are better vetted, less risky, and have higher individual qualities.

Experiment: Start funding more female founders.

If we then observe: The numbers start to even out, then there is no fundamental difference. VC funding bias may have been the cause of the difference in success rate.

If we then observe: The numbers stay the same, then there is a fundamental difference and our assumption is flawed.

Rational choice: Start funding more female founders. This either removes a bias (levels the playing field), or increases your profit (funding more potentially successful founders).

PG should of course not use an hypothesis to prove an assumption (experiment/probing is needed for verification). But also: The possibility of an uneven distribution should not invalidate such an experiment (or PG's line of reasoning), it will merely bring it to light (the numbers would stay the same, thus we have shown that the difference is fundamental and not caused by a sampling bias).