Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Authors’ names have ‘astonishing’ influence on peer reviewers: study

nature.com

Post reply on HN
Print view View original

221–230 of 319 posts

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#221

Post by rusticpenn »

jonathanstrange wrote: post #213
epolanski wrote: post #209

As a former researcher with multiple high-impact publications I can 100% confirm it: - when I was a researcher in Italy for an unknown lab, all of our articles were generally very scrutinized and went through years of reviews before publication - when I worked in Michal Graetzel[1]'s laboratory things get published much more easily on more higher impact journals with less scrutiny. Not only most of the publications d…

I don't understand this. Don't you have blind peer review? In my discipline, most journals even have double blind peer review. I thought that's the standard.

You can easily find the authors based on the references they use and the topic.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#222

Post by chalst »

keithwinstein wrote: post #124

Double-anonymous/blind reviewing is completely standard in most areas of computer science, e.g. networking (SIGCOMM, NSDI, IMC, HotNets, MobiCom, MobiSys), systems (OSDI, SOSP, USENIX ATC, HotOS), security (Usenix Security, S&P), machine learning (NeurIPS and ICML), graphics and HCI (SIGGRAPH and CHI), and at least some top-tier theory conferences (like FOCS). So I think most computer scientists would agree with the…

Double-blinding is not common in PL theory and implementation.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#223

Post by rendall »

epolanski wrote: post #219
jonathanstrange wrote: post #213

Earlier quoted context omitted.

I don't understand this. Don't you have blind peer review? In my discipline, most journals even have double blind peer review. I thought that's the standard.

> Don't you have blind peer review? It's fundamentally impossible because when you publish the topics are going to be in a way or another niche enough that you know all of the people working on those topics and conferences make those circles even more public. Say that you are studying e.g. "helical peptides", which is a vague and broad term on one side, on the other one, all the people that work on specific aspects o…

But this is contradictory. Can you explain?

On the one hand are famous scientists who have wide influence, on the other hand are small academic circles working on obscure, niche topics where everyone knows each other so well that blind review is impractical.

How can one scientist be both so reknowned that their name tilts the balance of grant money unfairly away from other less known researchers, and at the same time their research is so specific that only a very few others can even comment on their work?

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#224

Post by minifridge »

Skimming through the article, it fails to mention that journals have some power to assess systematic reviewer biases based on their past history. If reviewers are systematically more permissive towards specific people's work, then good science is not their objective.

So, essentially it can be viewed as a diversity problem in the reviewer space. The journals have important responsibility on this and they can not present as neutral observers of this phenomenon where they say : "oh every peer reviewing model has problems " and blame reviewers ethos that they are choosing.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#225

Post by seydor »

This is not surprising of course. What has changed probably is that science over the past 20 years has become more "social", but not in a good way. Since there are so many good scientists everywhere, people get ahead by using any means to make their science more public, mostly through conferences, organizing conferences, pursuing various committees and subcommittees, befriending journal editors, the press, and of course social media plays a part in this, but not alone. Antisocial people will have a hard time.

Where to start here? What do we want the purpose of publishing and peer review to be? Out of all this publicity dance, which part gets distilled into the solid foundation of science? I do think this whole journals/publishing/conference/apply-for-funding thingy is a bit too ancient and incremental, and more radical solutions would be nice to try. I think fundamental to this is the structure of funding, do we really want small funds going to individual small PIs, or maybe more independence, or less independence ...

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#226

Post by [deleted] »

emacsen wrote: post #94

When I was an undergrad in college, I helped design a study on the role of gender perception in expertise. We had a piece of text that the subjects (undergrads) would read and rate the expertise of. We had the same text but we randomized whether the name would be a commonly male name, a commonly female name, or initials. We also randomized if they'd watch a clip of men's sports beforehand (men's basketball), women's…

[deleted]

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#227

Post by jwmoraes »

Isn’t the article confusing correlation with causation? I read like there’s a correlation between acceptance rate and author’s pedigree, but the article sounds like the pedigree is the cause of the high acceptance.

I’m no expert, so please correct if I’m wrong. But for example, how likely is that a Nobel prize winner produces worth-publishing research? Or how likely he is simply good at the skill of paper writing?

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#228

Post by andrepd »

erfgh wrote: post #214
emacsen wrote: post #94

When I was an undergrad in college, I helped design a study on the role of gender perception in expertise. We had a piece of text that the subjects (undergrads) would read and rate the expertise of. We had the same text but we randomized whether the name would be a commonly male name, a commonly female name, or initials. We also randomized if they'd watch a clip of men's sports beforehand (men's basketball), women's…

If it is known that most discoveries are made by males it is reasonable to give larger weight to new male-done research, if the only thing that is known about the researchers is their gender and if the reader is not enough of an expert on the subject matter to treat it 100% on its own merit. I don't see this "bias" as inefficient or counter-productive. It is just an artifact of the way you designed your flawed experi…

> To find any real bias you would have to assure the subjects that the researchers were equally accomplished.

What do you mean? I think you didn't understand the experiment. Subjects were shown the same text with, at random, a male name, female name, or initials. These names were not real researchers, nor familiar to the subjects.

> If it is known that most discoveries are made by males it is reasonable to give larger weight to new male-done research

I'm sure you can see the two different fallacies in this short sentence:

- There have been more male researchers than female, therefore more "discoveries" were made by men than women. No shit. This does not mean male researchers are better than female, obviously.

- There are more men being researchers therefore men's research should be given more weight therefore there are more men researchers therefore...

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#229

Post by Orothrim »

techas wrote: post #220
epolanski wrote: post #219

Earlier quoted context omitted.

> Don't you have blind peer review? It's fundamentally impossible because when you publish the topics are going to be in a way or another niche enough that you know all of the people working on those topics and conferences make those circles even more public. Say that you are studying e.g. "helical peptides", which is a vague and broad term on one side, on the other one, all the people that work on specific aspects o…

>It's fundamentally impossible because when you publish the topics are going to be in a way or another niche enough that you know all of the people working on those topics Even if you know the people in the field and you can guess the authors, double blind does not hurt. Actually, I don’t see any downside of double blind. Actually, names are very often used to let you out of the field if you are not part of the usual…

You can't see any downside of double blind?

There are many systems we use which provide anonymity, and they always result in some form of abuse. Such as phone numbers (scam calls), Internet (cyber harassment, scams, viruses), cryptocurrency (all the scams). Any system where humans can gain advantages from abusing anonymity they do so.

I'm not an academic so maybe there are already safe guards, but from my understanding people already cut any research into as many small papers as they can. Truly double blind peer review would likely encourage this. Maybe it would also further encourage people to steal research or peer review outside of their expertise.

The benefits may outweigh the negatives, but I'm sure people will find a way to abuse it too, consciously or not.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#230

Post by frankfrankfrank »

rendall wrote: post #223
epolanski wrote: post #219

Earlier quoted context omitted.

> Don't you have blind peer review? It's fundamentally impossible because when you publish the topics are going to be in a way or another niche enough that you know all of the people working on those topics and conferences make those circles even more public. Say that you are studying e.g. "helical peptides", which is a vague and broad term on one side, on the other one, all the people that work on specific aspects o…

But this is contradictory. Can you explain? On the one hand are famous scientists who have wide influence, on the other hand are small academic circles working on obscure, niche topics where everyone knows each other so well that blind review is impractical. How can one scientist be both so reknowned that their name tilts the balance of grant money unfairly away from other less known researchers, and at the same time…

It is likely a relative matter. “Famous scientists” still tend to be famous within a rather small subset of even other scientists who are in a similar niche, no matter how large that niche is. It is the intensity of engagement of that community that likely makes it nearly impossible to even not tell exactly who wrote something, simply by the writing style.

An analogy may be how heavy readers can immediately tell which author of their particular topic area they are reading, based on a short excerpt. It is what makes alternative noms de plumes almost impossible, especially now with systematic and documenting AI.

Post reply on HN