Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Authors’ names have ‘astonishing’ influence on peer reviewers: study

nature.com

Post reply on HN
Print view View original

151–160 of 319 posts

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#151

Post by 9wzYQbTYsAIc »

nemo44x wrote: post #122
emacsen wrote: post #94

When I was an undergrad in college, I helped design a study on the role of gender perception in expertise. We had a piece of text that the subjects (undergrads) would read and rate the expertise of. We had the same text but we randomized whether the name would be a commonly male name, a commonly female name, or initials. We also randomized if they'd watch a clip of men's sports beforehand (men's basketball), women's…

It’s one thing to observe a pattern in a certain subset of people. However, why do you think there is this apparent bias or perception? If there is a bias, is it good, bad, or neutral? Are there biases in other directions?

> If there is a bias, is it good, bad, or neutral? Are there biases in other directions?

Bias isn’t inherently good or bad - they are normative judgements of distance from rationality.

Some biases are probably evolutionary advantageous to the individual but harmful to the society or others.

Sticking with the group is great, except when it isn’t, for example.

Or, a “gut check” is great, but you can’t always make decisions based on your gut:

https://en.m.wikipedia.org/wiki/Affect_heuristic

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#152

Post by [deleted] »

keithwinstein wrote: post #124

Double-anonymous/blind reviewing is completely standard in most areas of computer science, e.g. networking (SIGCOMM, NSDI, IMC, HotNets, MobiCom, MobiSys), systems (OSDI, SOSP, USENIX ATC, HotOS), security (Usenix Security, S&P), machine learning (NeurIPS and ICML), graphics and HCI (SIGGRAPH and CHI), and at least some top-tier theory conferences (like FOCS). So I think most computer scientists would agree with the…

[deleted]

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#153

Post by Blikkentrekker »

insane_dreamer wrote: post #129
Blikkentrekker wrote: post #58

Earlier quoted context omitted.

It's not different from Reddit or H.N. in the sense that most people that vote never bothered to read the entire post and the same thing is true with peer review. This isn't simply the case in soft-science as the Bognanov affair showed. They skim, see if it looks okayish and give it their approval.

HN doesn't allow you to follow users--that makes a big difference in putting the focus more on the content than the users.

Peer-reviewers also don't follow particular names around; they see the name above the submission and it influences them, as on H.N. no doubt.

But it's far worse, even without a recognized names, most votes are cast without proper reading, and I'm fairly certain also by the least intelligent subsection given how often submissions are upvoted on H.N. and Reddit that are pure clickbait and demolished in the comments by people that actually read it. People that vote by and large only read the title or the first sentence and make up their mind from there.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#154

Post by anm89 »

themitigating wrote: post #90
anm89 wrote: post #38

Is anyone surprised by this?

Why weren't you?

Much of academia seems to have become an outwardly transparent no hold barred status battleground. So this just seems to exactly align with my expected outcome given that assumption.

I'm not saying there aren't people in academia who aren't driven by doing good research but I certainly don't see that as the driving force in the US academia.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#155

Post by nextlevelwizard »

jedberg wrote: post #121

It blows my mind that peer reviews aren't done blind. Not only is there this geneder bias but also bias of "I know that person they're famous I'm sure their work is good" or "I've rejected their paper in the past". Seems like an easy fix would be to make all peer reviews blind. In fact, for any study that gets federal funding, they should have to publish their hypothesis ahead of time into an escrow system, submit th…

Academics just like every other aspect of human life is full of politics and this includes peer reviews as well. Papers regularly get panned for variety of reasons that have nothing to do with actual content of the paper and maybe even worse sometimes because the reviewer happen to be working on something similar and want secure a bigger grant for themselves.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#156

Post by kjeetgill »

olivermuty wrote: post #136
jedberg wrote: post #121

It blows my mind that peer reviews aren't done blind. Not only is there this geneder bias but also bias of "I know that person they're famous I'm sure their work is good" or "I've rejected their paper in the past". Seems like an easy fix would be to make all peer reviews blind. In fact, for any study that gets federal funding, they should have to publish their hypothesis ahead of time into an escrow system, submit th…

I like where you come from, but a LOT of phds are so narrow in scope that its almost impossible to be anonymous because the set of peer reviewers is very small. Only the large wide fields like medicine and some variants of humaniora are big enough to do this.

Interesting, I hadn't thought of that. I wonder if that's okay though — if you're that familiar with someone and their work I'd expect your bias towards that specific person overrides any broad gender bias.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#158

Post by 9wzYQbTYsAIc »

Ma8ee wrote: post #141
keithwinstein wrote: post #124

Double-anonymous/blind reviewing is completely standard in most areas of computer science, e.g. networking (SIGCOMM, NSDI, IMC, HotNets, MobiCom, MobiSys), systems (OSDI, SOSP, USENIX ATC, HotOS), security (Usenix Security, S&P), machine learning (NeurIPS and ICML), graphics and HCI (SIGGRAPH and CHI), and at least some top-tier theory conferences (like FOCS). So I think most computer scientists would agree with the…

I’m confused about your second point: isn’t the reader the one who gains the most from reading the best papers, independently of who’s the author?

I think the parent commenter is saying in (2) that sometimes readers want to follow a particular author, like some cartoonists prefer to read XKCD rather than Dilbert, even though they generally study all cartoons of their genre.

I’ve often seen celebrity authors write a “Letter to the Journal”, for those kind of readers, and I think that that might be the safer way for big names to be recognized.

It may give the author space to collaborate with interested parties without necessarily overwhelming the peer-reviewed content.

At the same time, some authors are truly just prolific.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#159

Post by cycomanic »

jedberg wrote: post #137
hannob wrote: post #134

Earlier quoted context omitted.

> It blows my mind that peer reviews aren't done blind. I've co-authored a few papers and blind reviews have some surprising consequences, like discussions along the lines of "you can't say that you already wrote about this issue elsewhere and put a link in the paper, because that would unblind it". I was a bit uncomfortable with that, because I like to "cite my sources" (even if the source was myself in this case).…

Whoever told you that about your cites is probably right -- you shouldn't cite yourself while calling out that you are citing yourself. You should cite yourself the same you that site any other work. The second part is probably unavoidable. If you're working on something super specialized when all your reviewers are the six other people who work on it then sure, it'll be unblinded. Not much we can do there sadly.

I disagree, the style to write as a "neutral" observer (often writing in passive voice) is frowned upon nowadays for good reason.

Part of that is also that the information if the authors wrote a citation or not can be important to a reviewer. For example it is unfortunately quite common that authors publish results in a salami tactic to maximize the number of publications. There can be a significant difference in impact between a citation saying "this is important to work on" which is written by the authors and one which is written by someone else.

Generally we should not write to hide information, and that would include if the authors wrote other work that relates to the work being reviewed. We should not adjust our writing to doubleblind review (and I would argue the advice the author was given is wrong). Doubleblind review is imperfect anyway, I can often tell who the authors are just by topic and e.g. writing and figure styles, so if a reviewer really wants to know the authors they can. We should still do double blind though.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#160

Post by kuwoze »

emacsen wrote: post #94

When I was an undergrad in college, I helped design a study on the role of gender perception in expertise. We had a piece of text that the subjects (undergrads) would read and rate the expertise of. We had the same text but we randomized whether the name would be a commonly male name, a commonly female name, or initials. We also randomized if they'd watch a clip of men's sports beforehand (men's basketball), women's…

No post body was provided.
Post reply on HN