Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Blind Reviews Experiment to remove code reviewer bias on Bugzilla and GitHub

addons.mozilla.org

Post reply on HN
Print view View original

11–20 of 20 posts

Re: Blind Reviews Experiment to remove code reviewer bias on Bugzilla and GitHub

#11

Post by legostormtroopr »

ubernostrum wrote: post #10
Arete31415 wrote: post #6

FYI, when orchestras started doing blind auditions, the numbers of women hired skyrocketed. Even though you'd think music would be the one area where what a person looked like & whether they were male or female would be irrelevant. I think this is a great idea.

This article is relevant for the discussion: https://peerj.com/articles/cs-111/ They explored a lot of factors to do with gender in a data set of several years' worth of public pull requests on GitHub. One of the more interesting findings was that women's pull requests get accepted more often, but that a couple factors can strongly affect that, including whether the person reviewing the PR knows the author of the PR…

This doesn't necessarily imply causation - that being a woman causes people to accept your pull requests.

Identification online is totally voluntary, so perhaps there is something about people who feel the need to specify their gender online, when it isn't necessary (such as when coding) that causes their pull requests to be rejected more.

Re: Blind Reviews Experiment to remove code reviewer bias on Bugzilla and GitHub

#12

Post by zilchers »

TheChaplain wrote: post #4

This seems like a problem that needs a cultural solution, not a technical one? Because knowing who submitted the code, I can better manage my time. E.g. if the submitter is known for producing high quality code, I only need to spend minimal time browsing through it. But, if the submitter is known to cut corners I can prepare myself to spend more time and review each line carefully because I most certainly will find i…

But isn’t that sort of the issue? If you make assumptions about overall code quality, you’ll probably let smaller stylistic things slide because “this is a good coder.” I think about this in terms of some of the more contentious project (think bitcoin, this would be great for removing some of the personality clashes there).

Re: Blind Reviews Experiment to remove code reviewer bias on Bugzilla and GitHub

#14

Post by true_religion »

legostormtroopr wrote: post #11
ubernostrum wrote: post #10

Earlier quoted context omitted.

This article is relevant for the discussion: https://peerj.com/articles/cs-111/ They explored a lot of factors to do with gender in a data set of several years' worth of public pull requests on GitHub. One of the more interesting findings was that women's pull requests get accepted more often, but that a couple factors can strongly affect that, including whether the person reviewing the PR knows the author of the PR…

This doesn't necessarily imply causation - that being a woman causes people to accept your pull requests. Identification online is totally voluntary, so perhaps there is something about people who feel the need to specify their gender online, when it isn't necessary (such as when coding) that causes their pull requests to be rejected more.

People on GitHub use their photos as their avatars so it's truly trivial to determine gender.

Re: Blind Reviews Experiment to remove code reviewer bias on Bugzilla and GitHub

#15

Post by nerdponx »

legostormtroopr wrote: post #8
Arete31415 wrote: post #6

FYI, when orchestras started doing blind auditions, the numbers of women hired skyrocketed. Even though you'd think music would be the one area where what a person looked like & whether they were male or female would be irrelevant. I think this is a great idea.

Except sometimes blind hiring doesn't go the direction you are hoping. http://www.abc.net.au/news/2017-06-30/bilnd-recruitment-tria...

Article didn't stop to get into why that happened. Were there just more qualified men in the applicant pool than women?

Re: Blind Reviews Experiment to remove code reviewer bias on Bugzilla and GitHub

#16

Post by legostormtroopr »

true_religion wrote: post #14
legostormtroopr wrote: post #11

Earlier quoted context omitted.

This doesn't necessarily imply causation - that being a woman causes people to accept your pull requests. Identification online is totally voluntary, so perhaps there is something about people who feel the need to specify their gender online, when it isn't necessary (such as when coding) that causes their pull requests to be rejected more.

People on GitHub use their photos as their avatars so it's truly trivial to determine gender.

Some people do use their photos. Many people don’t. It’s not “trivial” to determine anyone’s gender on GitHub.

Re: Blind Reviews Experiment to remove code reviewer bias on Bugzilla and GitHub

#17

Post by true_religion »

legostormtroopr wrote: post #16
true_religion wrote: post #14

Earlier quoted context omitted.

People on GitHub use their photos as their avatars so it's truly trivial to determine gender.

Some people do use their photos. Many people don’t. It’s not “trivial” to determine anyone’s gender on GitHub.

I had meant to say "many" people---as in many people in my experience.

To me, it's not very surprising as in many countries your picture is customarily included in your curriculum vitae, and as Github is seen as a professional site akin to LinkedIn, people simply put up their pictures there as well.

This is anecdotal, but it's very rare for me to interact with someone on Github and have no idea as to their real name, gender, and face.

---

I'll also add, there does seem to be cultural divide between us---as noted that you believe that people have to "feel a need" to add any information (photo/real name) that would show their gender when it isn't "necessary".

Whereas I feel that "real life" identification is the default, and pseudo-anonymity is a considered choice.

Re: Blind Reviews Experiment to remove code reviewer bias on Bugzilla and GitHub

#18

Post by UncleMeat »

legostormtroopr wrote: post #11
ubernostrum wrote: post #10

Earlier quoted context omitted.

This article is relevant for the discussion: https://peerj.com/articles/cs-111/ They explored a lot of factors to do with gender in a data set of several years' worth of public pull requests on GitHub. One of the more interesting findings was that women's pull requests get accepted more often, but that a couple factors can strongly affect that, including whether the person reviewing the PR knows the author of the PR…

This doesn't necessarily imply causation - that being a woman causes people to accept your pull requests. Identification online is totally voluntary, so perhaps there is something about people who feel the need to specify their gender online, when it isn't necessary (such as when coding) that causes their pull requests to be rejected more.

That is possible but without some data, any data, to back that hypothesis up it becomes fairly useless. Science doesn't really work by dismissing papers by throwing out unsupported alternative explanations for data.

Re: Blind Reviews Experiment to remove code reviewer bias on Bugzilla and GitHub

#19

Post by tdb7893 »

legostormtroopr wrote: post #8
Arete31415 wrote: post #6

FYI, when orchestras started doing blind auditions, the numbers of women hired skyrocketed. Even though you'd think music would be the one area where what a person looked like & whether they were male or female would be irrelevant. I think this is a great idea.

Except sometimes blind hiring doesn't go the direction you are hoping. http://www.abc.net.au/news/2017-06-30/bilnd-recruitment-tria...

Yeah, doing blind hiring won't fix most biases. People's careers are reflective of the oppourtunites that they have had and blind hiring seems like it has the potential to just reinforce existing societal biases instead of eliminating them.

Re: Blind Reviews Experiment to remove code reviewer bias on Bugzilla and GitHub

#20

Post by blablabla123 »

legostormtroopr wrote: post #11
ubernostrum wrote: post #10

Earlier quoted context omitted.

This article is relevant for the discussion: https://peerj.com/articles/cs-111/ They explored a lot of factors to do with gender in a data set of several years' worth of public pull requests on GitHub. One of the more interesting findings was that women's pull requests get accepted more often, but that a couple factors can strongly affect that, including whether the person reviewing the PR knows the author of the PR…

This doesn't necessarily imply causation - that being a woman causes people to accept your pull requests. Identification online is totally voluntary, so perhaps there is something about people who feel the need to specify their gender online, when it isn't necessary (such as when coding) that causes their pull requests to be rejected more.

Yeah but I mean it's quite plausible. There aren't many woman in IT, so I guess those who are, try to be especially careful, efficient etc. On the other hand there are these biases...

In fact this matches an observation an acquaintance of mine of is Post doc in the Physics department made. He is giving courses at University for small groups, that go with the lecture course where students get there corrected exercises and present them. So he said that women in those courses are more or less the only ones motivated and actually being interested in the stuff and that the exercises are also better.

I guess this doesn't have much to do with biology and Google Echo chambers etc but rather that people are aware of biases and counteract them.

Post reply on HN