Live data from Hacker News

Authors’ names have ‘astonishing’ influence on peer reviewers: study

nature.com

21–30 of 319 posts

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#21
That’s what we want in society right? Perks for doing hard work and then being recognized for it. That’s what “street cred” is. Maintaining a reputation is hard work and the fast pass and acknowledgment from the broader society is your reward.

As in this person has more than proved himself, let’s not vet him as much.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#22

Earlier quoted context omitted.

If you want a review to be truly double blind, you have to start censoring the paper -- things like "in previous work [1-10] we showed..." makes it obvious who the authors are even if you remove the names from the top of the paper.

You do not have to censor: you simply refer to yourselves in the third person to make your shared identity plausibly deniable. A work that is worthy of being of publication should largely be able to stand on its own anyway. This may have the effect of reducing minimum publishable unit (MPU) CV spam papers as a bonus.

I'm talking about editors censoring papers. Authors could avoid this problem by writing differently, but as a reader I'd prefer they don't; knowing that a body of work is closely related is very useful, especially when it comes to deciding which references to follow up.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#23

Earlier quoted context omitted.

You do not have to censor: you simply refer to yourselves in the third person to make your shared identity plausibly deniable. A work that is worthy of being of publication should largely be able to stand on its own anyway. This may have the effect of reducing minimum publishable unit (MPU) CV spam papers as a bonus.

I'm talking about editors censoring papers. Authors could avoid this problem by writing differently, but as a reader I'd prefer they don't; knowing that a body of work is closely related is very useful, especially when it comes to deciding which references to follow up.

The double blind process only requires that this be done during the review drafts. The final camera-ready version for publication can use the first person without compromising the integrity of the review process. I don't think the editors would remove self-referential text from a final draft after the paper had been accepted.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#24

It seems to me that some subjects are so specialized that the group that makes up one's 'peers' is fairly small. How do they have real anonymized reviews when it becomes easy to recognize the writings of the author? The more papers that someone writes, the easier it would be for their peers to recognize the writing style and other quirks that would give the author away.

This happens all the time in the field I'm in (phages, phage therapy) and the same names appear over and over. Peer review are anonymous but you can easily tell who's reviewed your paper or grant.

I'm not sure how to fix this problem though. In our phage newsletter we try to avoid using names and universities to focus on the paper/topic/finding itself, but I keep finding myself looking at author names and affiliations before diving into any paper.

I know it's "wrong" and I recognize myself doing it, but I still do it all the time.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#25
Fairness and efficiency are often at odds. As a reviewer, I was sometimes asked to review papers from unknown authors at unknown institutions. My level of effort was likely proportional to how well I knew the quality of the institution from which the paper came. There is a great deal of time involved in understanding a paper well enough to do a full review.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#26

This reminds me of a bias I saw on Quora, where some popular contributor gets tons of upvotes by fans regardless of whether their answer is any good or not (often you have to dig deep into the comments to find out that it's not a good answer). (I should qualify that I don't know if it's still that way; I started using it when it was first launched and then quit 6-7 years ago once it started turning into a fanboi and…

Well, that's because Quora has distribution mechanics like the home feed and email digests that can amplify content by popular contributors. It's not like Reddit or HN or StackExchange where answers/comments compete directly against each other on the question page. I don't think the situation is analogous.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#27

That’s what we want in society right? Perks for doing hard work and then being recognized for it. That’s what “street cred” is. Maintaining a reputation is hard work and the fast pass and acknowledgment from the broader society is your reward. As in this person has more than proved himself, let’s not vet him as much.

This is literally how one of the biggest scandals in academic publishing occurred[1], so... no? Maybe it's a nice way to treat experts when asking them for their opinion, they are experts after all, but we absolutely don't want to give celebrities more leeway than unknowns in research papers where new claims are being made. You better have done the work properly, again, and again, and again. No free passes, no blind trust.

[1] https://en.wikipedia.org/wiki/Diederik_Stapel

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#28

Earlier quoted context omitted.

If you want a review to be truly double blind, you have to start censoring the paper -- things like "in previous work [1-10] we showed..." makes it obvious who the authors are even if you remove the names from the top of the paper.

You do not have to censor: you simply refer to yourselves in the third person to make your shared identity plausibly deniable. A work that is worthy of being of publication should largely be able to stand on its own anyway. This may have the effect of reducing minimum publishable unit (MPU) CV spam papers as a bonus.

Usually if you count who has more citations you get the author of the paper (or the advisor, or the leader of the team). Sometimes it's a lifehack to get more citations, but most of the times is just natural because someone of the team was working in tool X and other member of the team in tool Y and now you are adjusting all the details to make X and Y work together and get a new result.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#29
I would expect a much higher than sixfold increase in paper acceptances by winning the Nobel Prize. Why wouldn’t there be a massive lift? Even if you can’t personally see why this result is important you know that one of the biggest contributors to the field thinks it’s interesting. That’s a pretty credible signal in a noisy world.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#30

How is double blind not the norm? there is just too much room for bias

If you want a review to be truly double blind, you have to start censoring the paper -- things like "in previous work [1-10] we showed..." makes it obvious who the authors are even if you remove the names from the top of the paper.

There are some computer science conferences that do this. If you cite your prior work, you are supposed to blank out the names in the citation and then attach an anonomyzed version of the prior work so that the reviewers can reference it if necessary. If the paper is accepted, you de-anonomyze the citation in the camera ready version.
Post reply on HN