Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Authors’ names have ‘astonishing’ influence on peer reviewers: study

nature.com

Post reply on HN
Print view View original

131–140 of 319 posts

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#131

Post by SassyThrowaway »

raister wrote: post #101
Test0129 wrote: post #97

Very common for anyone who has ever spent time in academia. When I was still in grad school I noticed a trend in some major mathematical and CS journals where a lot of time it would be the same N people writing articles. This was driven home when my advisor told me "without such and such's name this paper has no chance of making it past review". Non-sense. However, for all the people that say "trust the science" this…

First of all, Knuth would never publish something "garbage" because his reputation would be on the line for everyone to read (forever).

The point is that a famous name gives an edge in peer review to a (potentially mediocre) paper written primarily by somebody else with the famous name appended.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#133

Post by stevebmark »

Also relevant: "Cognitive ease" from "Thinking Fast And Slow." We often attribute information as higher quality if we're able to recall related information with more ease. Importantly, this happens regardless of the quality of the information. One study they point out is people exposed to random nonsensical words later ranked the words they saw the most frequently as being "better" than ones they saw less.

It sounds like a related bias of automatic thinking here. There's some cognitive ease in recognizing the author, so your bias kicks in that this is better than if you didn't recognize the author.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#134

Post by hannob »

jedberg wrote: post #121

It blows my mind that peer reviews aren't done blind. Not only is there this geneder bias but also bias of "I know that person they're famous I'm sure their work is good" or "I've rejected their paper in the past". Seems like an easy fix would be to make all peer reviews blind. In fact, for any study that gets federal funding, they should have to publish their hypothesis ahead of time into an escrow system, submit th…

> It blows my mind that peer reviews aren't done blind.

I've co-authored a few papers and blind reviews have some surprising consequences, like discussions along the lines of "you can't say that you already wrote about this issue elsewhere and put a link in the paper, because that would unblind it". I was a bit uncomfortable with that, because I like to "cite my sources" (even if the source was myself in this case).

This also points to another issue: The more specialized the paper is the less likely the blinding will work. If you know a field by heart then you know who works on what, and probably can guess most authors based on that.

I'm not saying I'm against blind review, but while it sounds obvious, it has some issues in practice.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#135

Post by ganesh7 »

bitexploder wrote: post #112
emacsen wrote: post #94

When I was an undergrad in college, I helped design a study on the role of gender perception in expertise. We had a piece of text that the subjects (undergrads) would read and rate the expertise of. We had the same text but we randomized whether the name would be a commonly male name, a commonly female name, or initials. We also randomized if they'd watch a clip of men's sports beforehand (men's basketball), women's…

Another interesting bias along those lines: https://en.m.wikipedia.org/wiki/Women-are-wonderful_effect I imagine these biases swing in all sorts of directions depending on the context. Some are intuitive, many are not.

good point.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#136

Post by olivermuty »

jedberg wrote: post #121

It blows my mind that peer reviews aren't done blind. Not only is there this geneder bias but also bias of "I know that person they're famous I'm sure their work is good" or "I've rejected their paper in the past". Seems like an easy fix would be to make all peer reviews blind. In fact, for any study that gets federal funding, they should have to publish their hypothesis ahead of time into an escrow system, submit th…

I like where you come from, but a LOT of phds are so narrow in scope that its almost impossible to be anonymous because the set of peer reviewers is very small. Only the large wide fields like medicine and some variants of humaniora are big enough to do this.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#137

Post by jedberg »

hannob wrote: post #134
jedberg wrote: post #121

It blows my mind that peer reviews aren't done blind. Not only is there this geneder bias but also bias of "I know that person they're famous I'm sure their work is good" or "I've rejected their paper in the past". Seems like an easy fix would be to make all peer reviews blind. In fact, for any study that gets federal funding, they should have to publish their hypothesis ahead of time into an escrow system, submit th…

> It blows my mind that peer reviews aren't done blind. I've co-authored a few papers and blind reviews have some surprising consequences, like discussions along the lines of "you can't say that you already wrote about this issue elsewhere and put a link in the paper, because that would unblind it". I was a bit uncomfortable with that, because I like to "cite my sources" (even if the source was myself in this case).…

Whoever told you that about your cites is probably right -- you shouldn't cite yourself while calling out that you are citing yourself. You should cite yourself the same you that site any other work.

The second part is probably unavoidable. If you're working on something super specialized when all your reviewers are the six other people who work on it then sure, it'll be unblinded. Not much we can do there sadly.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#138

Post by jedberg »

olivermuty wrote: post #136
jedberg wrote: post #121

It blows my mind that peer reviews aren't done blind. Not only is there this geneder bias but also bias of "I know that person they're famous I'm sure their work is good" or "I've rejected their paper in the past". Seems like an easy fix would be to make all peer reviews blind. In fact, for any study that gets federal funding, they should have to publish their hypothesis ahead of time into an escrow system, submit th…

I like where you come from, but a LOT of phds are so narrow in scope that its almost impossible to be anonymous because the set of peer reviewers is very small. Only the large wide fields like medicine and some variants of humaniora are big enough to do this.

If you're working on something super specialized when all your reviewers are the six other people who work on it then sure, it'll be unblinded. Not much we can do there sadly. But most people aren't working on something that specialized, and most would probably benefit from having reviewers in adjacent fields and not just the other experts in their field. So at least those adjacent ones would maybe be anonymous.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#139

Post by seanmcdirmid »

jedberg wrote: post #121

It blows my mind that peer reviews aren't done blind. Not only is there this geneder bias but also bias of "I know that person they're famous I'm sure their work is good" or "I've rejected their paper in the past". Seems like an easy fix would be to make all peer reviews blind. In fact, for any study that gets federal funding, they should have to publish their hypothesis ahead of time into an escrow system, submit th…

Peer reviews are usually blind (authors don’t know who their reviewers are). What you are referring to is double blind (reviewers don’t know who the authors are). The challenge with double blind is anonymizing content that often relies on cited prior work.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#140

Post by 9wzYQbTYsAIc »

emacsen wrote: post #94

When I was an undergrad in college, I helped design a study on the role of gender perception in expertise. We had a piece of text that the subjects (undergrads) would read and rate the expertise of. We had the same text but we randomized whether the name would be a commonly male name, a commonly female name, or initials. We also randomized if they'd watch a clip of men's sports beforehand (men's basketball), women's…

That reminds me of a bias study where the researchers were investigating the effects of stereotypes.

They told one group of Asian women that “women are worse at math than men” and another group of Asian women that “Asians are better at math than non-Asians”. Both are common stereotypes.

They then measured how well the two groups did on subsequent math exercises.

Interestingly, they found that the latter group (positive stereotype) performed better than those in the former group (negative stereotype).

As I recall, other research found similar results with Black males and golf scores when told “white people are better at golf” as opposed to “black people are better athletes”.

Not only do stereotypes influence our perceptions of “the other”, but they influence the performance of the other.

Post reply on HN