Live data from Hacker News

A Way to Detect Bias

paulgraham.com

101–110 of 224 posts

Re: A Way to Detect Bias

#101
Okay, PG has an hypothesis test.

There's a large literature for that, e.g.,

E. L. Lehmann, Testing Statistical Hypotheses.

E. L. Lehmann, Nonparametrics: Statistical Methods Based on Ranks.

Sidney Siegel, Nonparametric Statistics for the Behavioral Sciences.

In this case, PG will be more interested in the non-parametric case, i.e., distribution-free where we make no assumptions about probability distributions.

We start an hypothesis test with an hypothesis, commonly called the null hypothesis which is an assumption that there is no effect or, in PG's case, no bias. Then with that assumption, we are able to do some probability calculations.

Then we look at the real data and calculate the probability of, say, the evidence of bias being as large as we observed. If that probability is small, say, less than 1%, then we reject the null hypothesis, that is, reject the assumption of no bias, and conclude that the null hypothesis is false and that there is bias. The role of the assumption about the sample is so that we know that the problem is bias and not something about the sample.

In hypothesis testing, about all that matters are just two numbers -- the probability of Type I error and that of Type II error. We want both probabilities to be as low as possible.

Type I Error: We reject the null hypothesis when it is true, e.g., we conclude bias when there is none.

Type II Error: We fail to reject (i.e., we accept) the null hypothesis when it is false.

When looking for bias, Type I error can be called a false alarm of bias, and Type II error can be called a missed detection of bias.

In PGs case, suppose we have 100 startups and five of those have women founders. Suppose for each of the startups we have the data from "their subsequent performance is measured".

Our null hypothesis is that the expected performance of the women is the same as that of the men.

So, let's find those two averages and take the difference, say, the average of the women less the average of the men.

PG says if this difference is positive, then there was bias, but PG has not given us any estimate of the probability of Type I error, that is, of the probability (or rate) of a false alarm.

I mean we don't want to get First Round Capital in trouble with Betty Friedan, Gloria Steinem, Marissa Mayer, Sheryl Sandberg, Hillary Clinton, Ivanka Trump, or Lady Gaga unjustly! :-).

Let's call this difference our test statistic.

So, let's find the probability of a false alarm:

So, let's put all 100 measurements in a pot, stir the pot vigorously (we can use a computer for this), pull out five numbers and average, pull out the other 95 numbers and average, take the difference in the two averages, that of the five less that of the 95, and do this, say, 1000 times. Ah, computers are cheap; let's be generous and do this 10,000 times.

For a random number, how about starting with a 32 bit integer, with appropriately long precision arithmetic multiply by 5^15, add 1, take modulo 2^47, and scale as we want?

So, we get an empirical distribution of these differences, from the five less the 95. Looking at the distribution, we see what the probability is of getting a difference as high or high or higher than our test statistic. If that probability is low, say, 1% or less, then we reject the null hypothesis of no bias and conclude bias with our estimate of probability of Type I error 1% or less.

If with the 1% we reject, then it looks like First Round has done a transgression, will get retribution from Betty, et al., and needs to seek redemption and Betty, et al., are happy to have their suspicions confirmed. Else First Round looks like the good guys, are "certified statistically fair to women", may get more deal flow from women, and Betty, et al., can be happy that First Round is so nice!

Notice that either way Betty, et al., are "happy". That's called "happy women, happy life"! Or, heads, the women win, tails they lose, and in no event is there a huge crowd of angry women in front of First Round's offices with a bonfire of lingerie screaming "bias"!

When we reject the null hypothesis, we want to know that the reason was men versus women and not something else, e.g., a biased sample. So here is where we use our assumption of independence with the same mean.

Now we have a handle on Type I error.

Here we have done a non-parametric statistical hypothesis test, i.e., have made no assumptions, except the means, about the distributions of the male/female CEO performance measurements.

And we can select our desired false alarm rate in advance and get that rate almost exactly.

For Type II error, that is more difficult.

Bottom line, what we really want is, for whatever rate of false alarms we are willing to tolerate, the lowest rate of missed detections we can get.

Can we do that? With enough more data, yup. There is a classic result due to J. Neyman (long at Berkeley) and K. Pearson (early in statistics) that shows how.

How? Regard false alarm rate as money and think of investing in SF real estate. We put our money done on the opportunities with highest expected ROI until we have spent all our money. Done. For details, an unusually general proof can follow from the Hahn decomposition from the Radon-Nikodym theorem in measure theory, e.g., Rudin, Real and Complex Analysis. Right, in the discrete case, we have a knapsack problem, known to be in NP-complete.

What we have done with our pot stirring is called resampling, and for more such look for B. Efron, long at Yale, and P. Diaconis, once at Harvard, now long at Stanford.

Tom, with a reputation as a hacker, likes to work late, say, till 2 AM. So, we look at the intrusion alerts each minute between 2 AM and 3 AM (something like the performance of the women) and compare with those of the other minutes of 24 hours (like the performance of the men) much as above and ask if Tom is trying to hack the servers.

Or, we have a server farm and/or a network, and we want to detect problems never seen before, e.g., zero day problems. So, we have no data at all on the problems we are trying to detect because we have never seen any of those before.

So, to do a good job, let's pick some system we want to monitor and for that system, get data on, say, each of 10 variables at, say, 20 times a second. Now what?

Our work with bias in women venture applications used just one number for our measurement and test statistic. So we were uni-dimensional. Here we have 10 numbers and need to be multi-dimensional.

Well, in principle we should be able to do much better (pair of Type I and Type II error rates) with 10 numbers than just one. The usual ways will require us to have, with our null hypothesis, the probability distribution of the 10 numbers, but can only get something like that from smoking funny stuff -- not even big data is that big.

So, we want to need no assumptions about distribution, that is, be distribution-free.

So, we want some statistical a hypothesis test that is both multi-dimensional and distribution free.

Can we do that? Yup.

"You mean you can select false alarm rate in advance and get that rate essentially exactly, as in PG's bias example?" Yup.

"Could that be used in a real server farm or network to detect zero day problems -- security, performance, hard/software failures, system management errors?" Yup -- just what it was invented for.

"Attempted credit card fraud?" Ah, once a guy in an audience thought so!

How? Ah, sadly there is no more room in this post!

What else might we do with hypothesis tests? Well, look around at, right, big data or just small data.

Do we have a case of big data analytics or artificial intelligence (AI)?

Ah, I've given a sweetheart outline of statistical hypothesis testing, and now you are suggesting some things really low grade? Where did I go wrong to deserve such an insult?

Re: A Way to Detect Bias

#102

On a simple mathematical basis, this is false. Consider two groups of candidates for a scholarship, A and B. We want to select all candidates that have an 80% or better chance of graduation. Group A comes from a population where the chance of graduation is distributed uniformly from 0% to 100% and group B is from one where the chance is distributed uniformly from 10% to 90%, with the same average but less variation i…

Graham's intuition is assuming equality of the two distributions. As I noted in a different comment here, you can pretty easily fix Graham's test. Compute min(accepted a) and min(accepted B) instead of the means. In your example, the min of the accepted distributions would both work out to be 80%.

You can only rarely calculate min(accepted a) in the actual world. In this example, the college learns no distributions; they only know whether the student passed or failed.

Re: A Way to Detect Bias

#103

On a simple mathematical basis, this is false. Consider two groups of candidates for a scholarship, A and B. We want to select all candidates that have an 80% or better chance of graduation. Group A comes from a population where the chance of graduation is distributed uniformly from 0% to 100% and group B is from one where the chance is distributed uniformly from 10% to 90%, with the same average but less variation i…

So in this particular case, assuming that First Round's sample size is significant, it may just be that the female founders who seek them out are just on average better than the male ones? I suppose that if women think that the selection process is biased against them, and most do (and it may be) perhaps the less than excellent ones just don't apply, whereas that isn't true for males?

This seems plausible to me (though no less problematic, of course).

Re: A Way to Detect Bias

#104
I think I have a simpler counterexample to disprove pg's hypothesis than any other counterexample I've read in the comments. Suppose our goal is to admit the top 5 applicants with the following performances:

  A - 30,000
  A - 10,000
  A - 9,000
  B - 7,000
  B - 5,000   # Cutoff point below this line
  A - 4
  B - 3
  B - 2
Even though admitting the top 5 by score is perfectly fair, the applicants from group A perform better.

Re: A Way to Detect Bias

#105

On a simple mathematical basis, this is false. Consider two groups of candidates for a scholarship, A and B. We want to select all candidates that have an 80% or better chance of graduation. Group A comes from a population where the chance of graduation is distributed uniformly from 0% to 100% and group B is from one where the chance is distributed uniformly from 10% to 90%, with the same average but less variation i…

So in this particular case, assuming that First Round's sample size is significant, it may just be that the female founders who seek them out are just on average better than the male ones? I suppose that if women think that the selection process is biased against them, and most do (and it may be) perhaps the less than excellent ones just don't apply, whereas that isn't true for males?

The problem is different than that. It's that the measurement of performance is susceptible to differences in the conditional distribution of disparate groups, given that they were selected.

We can illustrate it by modifing the GP example to not include time at all, and make the metric perfect.

Suppose we are selecting for the next qualifying round for the Olympic 400m team, and select candidates if their 400m is under 50 seconds. Then, we measure performance -- immediately -- by a 400m run. We have two candidate pools: people who compete in the professional circuit, and everybody else. 100% of the former group who apply qualify, while only 10% of the others do.

Okay, so now we immediately measure the average 400m time of all professionals, versus average time of all amateurs who can beat 50 seconds. It's pretty reasonable to expect that the professionals might average closer to 45 seconds, while the other group might average around 48 (I'm not a 400m expert, the actual numbers might be off. WR is 43 seconds).

According to the article, we now conclude that our selection process is actually biased _against_ professionals! This is at the least very counter-intuitive. Maybe we provide both groups with coaching, and re-test after 6 months, a year, whatever. The professionals will certainly still outperform the amateurs. However, suppose one of the amateurs goes on to greatness, and wins. Wouldn't this obviously be biased against amateurs, according to our intuition?

Re: A Way to Detect Bias

#106

I don't understand his point about First Round Capital showing their female founders did better than companies without female founders. What does that show? How do we know that female founders aren't simply better? Or maybe women are scared of applying, so out of women, only the best apply? In that case, the mere idea that there is a bias can cause "pre-selection" bias. I lack the mathematics to prove this, but it se…

It's certainly possible that the observed bias isn't the fault of First Round Capital's selection process if the applying populations are different -- the "pre-selection" bias you're talking about.

If that's the case, First Round Capital could profitably benefit from encouraging more female founders to apply.

Re: A Way to Detect Bias

#107
post #93
post #85

Earlier quoted context omitted.

The problem here is language and what our actual objectives are. When people complain about bias, they are not really talking about mathematical bias, but about something else: Their idea of fairness. They are talking about discrimination. And when we are discussing that, we can't really think about whether rules are applied fairly or not, but whether the rules produce the outcomes that we want. Let's go for a ludicr…

Great comment. There are two types of fairness, (a) fair rules, and (b) fair outcome.

Favoring fair outcome over having fair rules is against everything I believe. We should not strive for a participation trophy culture.

Re: A Way to Detect Bias

#108

People, most of whom clearly are not that good at math, are being really harsh on Paul Graham. Graham is mostly right, but slightly incorrect. In particular, suppose group A has the distribution f(x) and B has the distribution g(x). If f(x) and g(x) are shaped significantly differently past the cutoff , then mean(H(x-C)f(x)) and mean(H(x-c)g(x)) might not agree even though there is no bias by construction. (Here H(x)…

> In practice, measure the weakest male and weakest female to be accepted in your sample set, or some similar approximation.

In practice, where do you get these definitive unbiased measurements of the expected return of investing in an individual (or graduation probability or some other score)?

Re: A Way to Detect Bias

#109
post #85

On a simple mathematical basis, this is false. Consider two groups of candidates for a scholarship, A and B. We want to select all candidates that have an 80% or better chance of graduation. Group A comes from a population where the chance of graduation is distributed uniformly from 0% to 100% and group B is from one where the chance is distributed uniformly from 10% to 90%, with the same average but less variation i…

The problem here is language and what our actual objectives are. When people complain about bias, they are not really talking about mathematical bias, but about something else: Their idea of fairness. They are talking about discrimination. And when we are discussing that, we can't really think about whether rules are applied fairly or not, but whether the rules produce the outcomes that we want. Let's go for a ludicr…

What surprised me a bit is that pg decided to use the word "bias" without any clarification, considering his background in computer science and AI.

Anyway, I think pg's whole argument is rather moot because the three assumptions that he states are incredibly difficult to measure (Part of the reason why it is very difficult to argue for or against affirmative actions without coming across as "biased").

Post reply on HN