Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Authors’ names have ‘astonishing’ influence on peer reviewers: study

nature.com

Post reply on HN
Print view View original

91–100 of 319 posts

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#91

Post by themitigating »

dangom wrote: post #5
j7ake wrote: post #4

Is it any different to podcasts, concerts, and books? People will choose what they listen, see, or read heavily based on who the performer is. A performer who has consistently given good content will obviously have a bigger pull than a nobody still trying to get their first break.

Podcasts, concerts and books are something we review subjectively. Scientific articles one would expect would be reviewed objectively.

The data is objective but the quality of the research is subjective. For example if my sample size was 150 for some study that's subjective whether it was sufficient of size

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#92

Post by hedora »

bpodgursky wrote: post #60
abdullahkhalids wrote: post #55

Earlier quoted context omitted.

Besides the sibling comment, in a lot of fields, people just know which group a paper is coming from even without doing any sort of searching or looking at preprints.

Yeah it's often pretty obvious that a paper extends / is incremental on a previous body of work (maybe the same code, maybe same dataset) such that it could only have come from one lab.

Yeah, but good CS conferences have diverse reviewer pools, including ones from industry or different subdisciplines. Such reviewers often haven't heard of the lab, or paid attention to which grants are currently funded, the movement of students into postdoc positions at other labs, etc, etc.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#93

Post by themitigating »

AussieWog93 wrote: post #53
hackerlight wrote: post #46

Earlier quoted context omitted.

Yes. The purpose of science is supposed to be finding the truth. The identical paper being rejected more often based on the celebrity of the author isn't that.

Honestly, this is the least of academia's problems. I've seen it from the inside. Probably read 80-100 widely-cited papers during my PhD (before dropping out), and maybe half a dozen of them were written by people who had any interest in discovering truth and pushing mankind forward. Seriously cannot overstate both the willful ignorance of established scientists, and the extent to which this is enforced onto the next…

Let me guess, the others wanted to push their political view and that's why I shouldn't trust experts?

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#94

Post by emacsen »

When I was an undergrad in college, I helped design a study on the role of gender perception in expertise.

We had a piece of text that the subjects (undergrads) would read and rate the expertise of. We had the same text but we randomized whether the name would be a commonly male name, a commonly female name, or initials.

We also randomized if they'd watch a clip of men's sports beforehand (men's basketball), women's sports beforehand (women's basketball), or no sports. And finally we randomized whether the person giving the subjects instructions would be a young woman (one of my classmates) or a young man (me).

Our small study showed what you'd expect- students- men and women students rated male writers more highly than female writers, and initials were right in the middle, though they tended to be more like responses for men.

The tester's gender made no difference that we found.

The sports thing made a measurable difference, but it didn't reverse the skew.

My 18/19 year old self was somewhat skeptical we'd find a difference, but I was totally wrong. It taught me a lot about bias and perception, which this study also shows.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#95

Post by themitigating »

nequo wrote: post #33
aussiesnack wrote: post #31

That an economist finds human behaviour that every normal human would expect "astonishing" is itself not remotely astonishing.

Peer review is not supposed to be a Twitter popularity contest so these numbers are indeed astonishing.

Right, nothing is really. The parent is saying that this is a common human flaw and exists everywhere

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#96

Post by themitigating »

quickthrower2 wrote: post #89
version_five wrote: post #52

Elon Musk tweets something stupid and it gets discussed. You or I do it and it's ignored. Celebrity is a thing, nothing odd going on

Discussed is not the same thing as reviewed though. I agree it is not odd. It is a good thing to know about though and be mindful of. It might be advantageous in closed circumstances, i.e. you can skim a review of a more trusted developer.

Who wasn't mindful that the concept of bias and popularity exist?

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#97

Post by Test0129 »

Very common for anyone who has ever spent time in academia. When I was still in grad school I noticed a trend in some major mathematical and CS journals where a lot of time it would be the same N people writing articles. This was driven home when my advisor told me "without such and such's name this paper has no chance of making it past review".

Non-sense. However, for all the people that say "trust the science" this is the science you're working with. It's politics all the way down just like everything else. For example, if Knuth decided to post some utter garbage chances are it would accepted based on his name alone.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#98

Post by quickthrower2 »

themitigating wrote: post #96
quickthrower2 wrote: post #89

Earlier quoted context omitted.

Discussed is not the same thing as reviewed though. I agree it is not odd. It is a good thing to know about though and be mindful of. It might be advantageous in closed circumstances, i.e. you can skim a review of a more trusted developer.

Who wasn't mindful that the concept of bias and popularity exist?

I think many people. Everyone knows. Not everyone is mindful.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#99

Post by hedora »

nequo wrote: post #50
hedora wrote: post #43

I'm surprised that there are still non double blind publications that count for tenure cases. This does explain some of the recent and embarrassing [lack of] retractions of Nature papers though.

It is nearly impossible to enforce double blind peer review because of the existence of online pre-prints (hosted on arXiv, SSRN, or just the authors' websites). A referee can copy and paste from the text into DuckDuckGo and find the authors.

I think you are overestimating how much reviewers care which grad student wrote which paper, or who their advisor is.

Reviewers are generally overworked (for other reasons), and the main goal is to be constructive and somehow not sound like an idiot (in front of the other reviewers, who might be subject experts and definitely know who you are!) in any of the pile of reviews you need to write after reading a pile of papers.

Also, reviewers declare conflicts ahead of paper assignments, which makes accidents less frequent.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#100

Post by themitigating »

Test0129 wrote: post #97

Very common for anyone who has ever spent time in academia. When I was still in grad school I noticed a trend in some major mathematical and CS journals where a lot of time it would be the same N people writing articles. This was driven home when my advisor told me "without such and such's name this paper has no chance of making it past review". Non-sense. However, for all the people that say "trust the science" this…

You trust science because it has the highest probability of being accurate. What's the alternative? Most people aren't capable of analysis scietific research to see if it's correct.

What makes something political? If new research shows that X product is dangerous and republicans, who that company donates to, come out and disagree with the research they just made it political. Now, someone like you, can just say "This research is political"

You've created a way to dismiss scientific research simply by disagreeing with it

Post reply on HN