Live data from Hacker News

Authors’ names have ‘astonishing’ influence on peer reviewers: study

nature.com

81–90 of 319 posts

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#81
post #43

I'm surprised that there are still non double blind publications that count for tenure cases. This does explain some of the recent and embarrassing [lack of] retractions of Nature papers though.

Peer reviews are blind, but in a lot of fields, once you've read enough papers it's obvious from subject and style who wrote the paper

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#82

We have to come to terms with this. Academia was bought and paid for long ago and the money was used to build an incredibly broken and overly political bureaucratic engine of scholarly and scientific work that doesn't get anywhere near as much peer-review scrutiny as it should and commands far more respect in politics and legal proceedings than we should allow. Universities and experts are the best we can do sometime…

The 2 academics I know aren’t bought and paid for. Either could make far more money in the private sector than they make now. They work in their fields out of love of research.

This was my impression of most of my professors years ago too, especially as we saw so many quit for high paying engineering jobs at companies year to year — the ones who stayed aren’t in it for the money

I suspect it’s the same at the top: most senior administrators I bet would make more in senior Fortune 500 jobs

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#83

We have to come to terms with this. Academia was bought and paid for long ago and the money was used to build an incredibly broken and overly political bureaucratic engine of scholarly and scientific work that doesn't get anywhere near as much peer-review scrutiny as it should and commands far more respect in politics and legal proceedings than we should allow. Universities and experts are the best we can do sometime…

Do you have any evidence of this?

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#84
post #70

This reminds me of a bias I saw on Quora, where some popular contributor gets tons of upvotes by fans regardless of whether their answer is any good or not (often you have to dig deep into the comments to find out that it's not a good answer). (I should qualify that I don't know if it's still that way; I started using it when it was first launched and then quit 6-7 years ago once it started turning into a fanboi and…

There was a study some time back where they set folks up in groups, then had them listen and vote on bands / music. Each group trended towards catapulting a single band / musician, but it always just depended on which one in the group got the momentum first. I wish I could find the study again but it's hard to google for. Pretty eye opening.

Salganik MJ, Dodds S, Watts DJ Experimental Study of Inequality and Unpredictability in an Artificial Cultural Market

https://www.science.org/doi/10.1126/science.1121066

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#85
post #30

Earlier quoted context omitted.

If you want a review to be truly double blind, you have to start censoring the paper -- things like "in previous work [1-10] we showed..." makes it obvious who the authors are even if you remove the names from the top of the paper.

There are some computer science conferences that do this. If you cite your prior work, you are supposed to blank out the names in the citation and then attach an anonomyzed version of the prior work so that the reviewers can reference it if necessary. If the paper is accepted, you de-anonomyze the citation in the camera ready version.

This does not necessarily help. In many fields the reviewers will be familiar with the prior work and therefore recognize it, anonymized or not.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#86

I'm not sure what we're supposed to find astonishing about academics (like most of humanity) being vulnerable to reputation biases in their reasoning. As in, that would be the null hypothesis. It would be astonishing if most academics overcame it. EDIT: apparently, the size of the effect (factor of six more likely to be accepted if it came from a Nobel Prizewinner) is larger than anticipated.

Are people in general bias based on reputation?

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#87
post #33

Earlier quoted context omitted.

Peer review is not supposed to be a Twitter popularity contest so these numbers are indeed astonishing.

Yet most non-economists would have predicted exactly this. If it's astonishing to you, well, perhaps you've had the misfortune to learn to think like an economist (ie. poorly & very muddy about normative vs actual).

It would be possible to respond to your comment in earnest but ad hominem like this is more of a psychological cushion than an invitation to genuine discussion.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#88

Every paper Albert Einstein submitted to anonymous peer review was rejected. (he had only one peer anonymous reviewed paper and it had an error)

I'm confused because of your second sentence but according to this site https://mindmatters.ai/2020/05/einsteins-only-rejected-paper...

". Albert Einstein only had one anonymous peer review in his career — and the paper was rejected2. This happened in 1936."

How is just one paper submitted anonymously being rejected an indication of a trend?

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#89

Elon Musk tweets something stupid and it gets discussed. You or I do it and it's ignored. Celebrity is a thing, nothing odd going on

Discussed is not the same thing as reviewed though. I agree it is not odd. It is a good thing to know about though and be mindful of. It might be advantageous in closed circumstances, i.e. you can skim a review of a more trusted developer.
Post reply on HN