Blind Reviews Experiment to remove code reviewer bias on Bugzilla and GitHub
1–10 of 20 posts
Re: Blind Reviews Experiment to remove code reviewer bias on Bugzilla and GitHub
#2Great idea in general. Software dev has an unnecessary amount of politics, and this could reduce it.
Re: Blind Reviews Experiment to remove code reviewer bias on Bugzilla and GitHub
#3Interesting idea, but everyone I've worked with has their own distinct flavors of how they code, so this wouldn't really hide identity much, not unless you didn't know the coder in the first place.
Re: Blind Reviews Experiment to remove code reviewer bias on Bugzilla and GitHub
#4This seems like a problem that needs a cultural solution, not a technical one?
Because knowing who submitted the code, I can better manage my time. E.g. if the submitter is known for producing high quality code, I only need to spend minimal time browsing through it.
But, if the submitter is known to cut corners I can prepare myself to spend more time and review each line carefully because I most certainly will find issues.
Re: Blind Reviews Experiment to remove code reviewer bias on Bugzilla and GitHub
#5This seems like a problem that needs a cultural solution, not a technical one? Because knowing who submitted the code, I can better manage my time. E.g. if the submitter is known for producing high quality code, I only need to spend minimal time browsing through it. But, if the submitter is known to cut corners I can prepare myself to spend more time and review each line carefully because I most certainly will find i…
I think it could be helpful for screening first time contributions at the very least. You're right that a cultural solution is necessary, but in the mean time one way to deal with bias when you know you are biased is to remove the ability for it to operate (when practicable).
Otherwise, you're stuck in this zone where you know you're liable to make bad decisions, but you rationalise to yourself that you can correct them. No matter how you slice it, it's never going to be possible to exactly correct for your biases, you're going to undershoot or overshoot. Better to minimize their scope of operation.
I really like this line of thinking because it crashes into this realm of philosophy where some people think you can feel your way to a good result, whereas the other approach is to initially intuit the just result, and rationally design a system that achieves it regardless of your instantaneous emotions. Of course, such systems thinking is very very sensitive to getting the right intuition up front, so it's very important to train it and build in feedback systems in case you start getting things you didn't expect.
Re: Blind Reviews Experiment to remove code reviewer bias on Bugzilla and GitHub
#6FYI, when orchestras started doing blind auditions, the numbers of women hired skyrocketed. Even though you'd think music would be the one area where what a person looked like & whether they were male or female would be irrelevant.
I think this is a great idea.
Re: Blind Reviews Experiment to remove code reviewer bias on Bugzilla and GitHub
#7This seems like a problem that needs a cultural solution, not a technical one? Because knowing who submitted the code, I can better manage my time. E.g. if the submitter is known for producing high quality code, I only need to spend minimal time browsing through it. But, if the submitter is known to cut corners I can prepare myself to spend more time and review each line carefully because I most certainly will find i…
There is room for both. Code review of high quality code is enlightening.
Some percentage should be blind. It’s a classic exploration vs exploitation trade off of your time.
Re: Blind Reviews Experiment to remove code reviewer bias on Bugzilla and GitHub
#8FYI, when orchestras started doing blind auditions, the numbers of women hired skyrocketed. Even though you'd think music would be the one area where what a person looked like & whether they were male or female would be irrelevant. I think this is a great idea.
Except sometimes blind hiring doesn't go the direction you are hoping.
http://www.abc.net.au/news/2017-06-30/bilnd-recruitment-tria...
Re: Blind Reviews Experiment to remove code reviewer bias on Bugzilla and GitHub
#9FYI, when orchestras started doing blind auditions, the numbers of women hired skyrocketed. Even though you'd think music would be the one area where what a person looked like & whether they were male or female would be irrelevant. I think this is a great idea.
Except sometimes blind hiring doesn't go the direction you are hoping. http://www.abc.net.au/news/2017-06-30/bilnd-recruitment-tria...
Yep. Also, the orchestra thing was 1952.
Re: Blind Reviews Experiment to remove code reviewer bias on Bugzilla and GitHub
#10FYI, when orchestras started doing blind auditions, the numbers of women hired skyrocketed. Even though you'd think music would be the one area where what a person looked like & whether they were male or female would be irrelevant. I think this is a great idea.
This article is relevant for the discussion:
https://peerj.com/articles/cs-111/
They explored a lot of factors to do with gender in a data set of several years' worth of public pull requests on GitHub. One of the more interesting findings was that women's pull requests get accepted more often, but that a couple factors can strongly affect that, including whether the person reviewing the PR knows the author of the PR is a woman:
For outsiders, we see evidence for gender bias: women’s acceptance rates drop by 12.0% when their gender is identifiable, compared to when it is not (χ2(df = 1, n = 16, 258) = 158, p