A Way to Detect Bias
paulgraham.com
A Way to Detect Bias
1–10 of 224 posts
Re: A Way to Detect Bias
#2I think the first footnote in this is extremely valid. It all depends on what performance metric the selection process is identifying compared to the performance metric you use to determine success.
I would suspect the larger issue is that people are probably much worse at identifying what performance metrics for selection convert to their respective performance for success.
Re: A Way to Detect Bias
#3[deleted]
Re: A Way to Detect Bias
#4Footnote one should be put more generally, biases in the performance metrics (either appearance for women vs ability for men, or how many words of the US national anthem someone knows for US citizens vs rest of the world) will cause this method to fail. Unfortunately, unbiased performance metrics are quite hard on single dimensions, let alone when moving to multi-dimensional metrics
Re: A Way to Detect Bias
#5Isn't every selection process based on experience, knowledge and/or mood and therefore baised?
Re: A Way to Detect Bias
#6I think the first footnote in this is extremely valid. It all depends on what performance metric the selection process is identifying compared to the performance metric you use to determine success. I would suspect the larger issue is that people are probably much worse at identifying what performance metrics for selection convert to their respective performance for success.
If students of Asian origin outperform the whole student body, can we conclude admissions folks are biased against students of Asian origin?
Re: A Way to Detect Bias
#7A related observation (which I've been making for a long time) is that the absence of mediocre women in positions of power is strong evidence of bias. Men can succeed when they're mediocre, but women have to be exceptional. Likewise for minorities.
Re: A Way to Detect Bias
#8Isn't every selection process based on experience, knowledge and/or mood and therefore baised?
He makes a few assumptions. But he is probably correct within those assumptions.
We assume all candidate pools are homogeneous. If all members of a certain subset A of a global population P is simply better at a task than any member of another subset B of the global population, we will see that this holds true for members of our sample as well. Thus, members of A in our sample will consistently outperform members of B in our sample. Does this mean there is a bias against A? Well, yes because if there wasn't then there would be fewer members of B or perhaps no members of B in our sample based on this result alone.
However, real life is not one-dimensional. Sometimes we need to consider other factors as well.
Re: A Way to Detect Bias
#9Actual real world example (and application of an antidote) of this basic idea:
https://en.wikipedia.org/wiki/Rooney_Rule
Re: A Way to Detect Bias
#10Graham's statement about the possible bias of First Round is unfounded. This was not any sort of a real study like Graham thinks and First Round clearly notes that. When the returns are as skewed as they are in venture capital (http://www.sethlevine.com/archives/2014/08/venture-outcomes-...), a small sample size and a simple analysis won't do. First Round even excluded their investment in Uber because it would skew the results too much.