Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Authors’ names have ‘astonishing’ influence on peer reviewers: study

nature.com

Post reply on HN
Print view View original

51–60 of 319 posts

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#51

Post by rgbrgb »

nequo wrote: post #50
hedora wrote: post #43

I'm surprised that there are still non double blind publications that count for tenure cases. This does explain some of the recent and embarrassing [lack of] retractions of Nature papers though.

It is nearly impossible to enforce double blind peer review because of the existence of online pre-prints (hosted on arXiv, SSRN, or just the authors' websites). A referee can copy and paste from the text into DuckDuckGo and find the authors.

In this case an easy 50% solution seems better than nothing. Some people would just comply with the guideline, some would not think of your workaround and it would make peer reviews a little more equitable even if many broke the rule.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#53

Post by AussieWog93 »

hackerlight wrote: post #46
j7ake wrote: post #4

Is it any different to podcasts, concerts, and books? People will choose what they listen, see, or read heavily based on who the performer is. A performer who has consistently given good content will obviously have a bigger pull than a nobody still trying to get their first break.

Yes. The purpose of science is supposed to be finding the truth. The identical paper being rejected more often based on the celebrity of the author isn't that.

Honestly, this is the least of academia's problems.

I've seen it from the inside. Probably read 80-100 widely-cited papers during my PhD (before dropping out), and maybe half a dozen of them were written by people who had any interest in discovering truth and pushing mankind forward.

Seriously cannot overstate both the willful ignorance of established scientists, and the extent to which this is enforced onto the next generation.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#54

Post by cortesoft »

hedora wrote: post #42
cozzyd wrote: post #37

Earlier quoted context omitted.

In my field that would be impossible... people work in collaborations and you know pretty much all of the experiments.

That's usually a consequence of too much nepotism, or some super expensive piece of equipment. Double blind helps the former, though not the latter. In a part of my field that is capital intensive, some well funded newcomers have recently invested a lot, and "broke in", while some incumbent testbeds went away; the "expensive equipment" problem is usually temporary if the field is expanding.

It also could just be a small specialty.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#55

Post by abdullahkhalids »

hedora wrote: post #43

I'm surprised that there are still non double blind publications that count for tenure cases. This does explain some of the recent and embarrassing [lack of] retractions of Nature papers though.

Besides the sibling comment, in a lot of fields, people just know which group a paper is coming from even without doing any sort of searching or looking at preprints.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#56

Post by bscphil »

insane_dreamer wrote: post #20

This reminds me of a bias I saw on Quora, where some popular contributor gets tons of upvotes by fans regardless of whether their answer is any good or not (often you have to dig deep into the comments to find out that it's not a good answer). (I should qualify that I don't know if it's still that way; I started using it when it was first launched and then quit 6-7 years ago once it started turning into a fanboi and…

It's probably inherent to any site that allows you to follow individuals. For Twitter that's not much of a problem because most people know better than to rely on Twitter for life advice or product reviews. Letterboxd, on the other hand, manifests exactly the problem you're talking about, and the whole point of the site is to assign purported ratings to film and TV shows.

Reddit and IMDb are mostly immune. Which is not to say that they're immune to all problems (astroturfing, brigading, personal biases, etc.).

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#57

Post by Blikkentrekker »

hedora wrote: post #43

I'm surprised that there are still non double blind publications that count for tenure cases. This does explain some of the recent and embarrassing [lack of] retractions of Nature papers though.

I've seen many arguments against double-blind which were very unconvincing, and they were all from the people that already had a big name, the ones that decide how journals operate.

In any case, this is the tip of the iceberg. The entire structure surrounding academic publication is absolutely ill-designed and few people in that world are even remotely interesting in safeguarding veracity.

The overwhelming majority of peer reviewed scientific results is infotainment for other scientists with no party having any material stake in the veracity of it, which is almost no one bothers to replicate because accuracy is irrelevant so long it be interesting to read.

The moment a company has bet a sizable stake on it's accuracy, then they suddenly check, and double check and have another party independently verify it because they do not want to loose money, obviously. But most science is nothing like that.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#58

Post by Blikkentrekker »

insane_dreamer wrote: post #20

This reminds me of a bias I saw on Quora, where some popular contributor gets tons of upvotes by fans regardless of whether their answer is any good or not (often you have to dig deep into the comments to find out that it's not a good answer). (I should qualify that I don't know if it's still that way; I started using it when it was first launched and then quit 6-7 years ago once it started turning into a fanboi and…

It's not different from Reddit or H.N. in the sense that most people that vote never bothered to read the entire post and the same thing is true with peer review. This isn't simply the case in soft-science as the Bognanov affair showed.

They skim, see if it looks okayish and give it their approval.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#59

Post by Blikkentrekker »

didgetmaster wrote: post #6

It seems to me that some subjects are so specialized that the group that makes up one's 'peers' is fairly small. How do they have real anonymized reviews when it becomes easy to recognize the writings of the author? The more papers that someone writes, the easier it would be for their peers to recognize the writing style and other quirks that would give the author away.

The nice part is that that can be gamed and should be gamed.

People should copy the writing style of famous auctor's then to expose the system and keep people honest because then they know there are copycats.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#60

Post by bpodgursky »

abdullahkhalids wrote: post #55
hedora wrote: post #43

I'm surprised that there are still non double blind publications that count for tenure cases. This does explain some of the recent and embarrassing [lack of] retractions of Nature papers though.

Besides the sibling comment, in a lot of fields, people just know which group a paper is coming from even without doing any sort of searching or looking at preprints.

Yeah it's often pretty obvious that a paper extends / is incremental on a previous body of work (maybe the same code, maybe same dataset) such that it could only have come from one lab.
Post reply on HN