Live data from Hacker News

Authors’ names have ‘astonishing’ influence on peer reviewers: study

nature.com

101–110 of 319 posts

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#101

Very common for anyone who has ever spent time in academia. When I was still in grad school I noticed a trend in some major mathematical and CS journals where a lot of time it would be the same N people writing articles. This was driven home when my advisor told me "without such and such's name this paper has no chance of making it past review". Non-sense. However, for all the people that say "trust the science" this…

First of all, Knuth would never publish something "garbage" because his reputation would be on the line for everyone to read (forever).

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#102
post #87

Earlier quoted context omitted.

Yet most non-economists would have predicted exactly this. If it's astonishing to you, well, perhaps you've had the misfortune to learn to think like an economist (ie. poorly & very muddy about normative vs actual).

It would be possible to respond to your comment in earnest but ad hominem like this is more of a psychological cushion than an invitation to genuine discussion.

In what world is a daft attempt to psychoanalyze your interlocutor not ad hominem?

My comment was snarky but not remotely playing the man. I meant it quite literally as a criticism of the thinking at issue. Economistic thought is riddled with confusion, and being surprised that scientists don't behave normatively (ie. not as those engaged in peer review are "supposed to") exhibits perfectly one variant of said confusion.

I'm not 'astonished' that non-blinded peer reviewers are influenced by social status. No-one I know would be surprised at all, let alone 'astonished'.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#103
post #43

I'm surprised that there are still non double blind publications that count for tenure cases. This does explain some of the recent and embarrassing [lack of] retractions of Nature papers though.

Even with double blind reviews, it’s often very clear who wrote the paper.

Authors regularly cite work by themselves or their team, so a statement like “In a previous study[1] we established a relationship between X and Y” renders double blind pointless.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#104

We have to come to terms with this. Academia was bought and paid for long ago and the money was used to build an incredibly broken and overly political bureaucratic engine of scholarly and scientific work that doesn't get anywhere near as much peer-review scrutiny as it should and commands far more respect in politics and legal proceedings than we should allow. Universities and experts are the best we can do sometime…

Do you have any evidence of this?

HN isn't a journal and so academic sources of evidence really are overkill. However, I'll throw you a bone. Go to your favorite major journal and start looking at the "conflicts of interest" and "grants" section.

Once you take money from someone (except for NIST, in my experience) you're basically beholden to try your hardest to get the results they're looking for. Some scientists are moral enough to still return bad results. A lot of scientists aren't. There's a lot of garbage out there, and the worse the journal, the more garbage it gets. Famously, Phillip Morris studies "passed" the scrutiny of several major journals. It's amazing what greasing a few palms will get you.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#105

Very common for anyone who has ever spent time in academia. When I was still in grad school I noticed a trend in some major mathematical and CS journals where a lot of time it would be the same N people writing articles. This was driven home when my advisor told me "without such and such's name this paper has no chance of making it past review". Non-sense. However, for all the people that say "trust the science" this…

First of all, Knuth would never publish something "garbage" because his reputation would be on the line for everyone to read (forever).

I mean it was an example I chose because the name recognition in HN. I would doubt Knuth would publish garbage. Then again, how would you know? If you've never been in academia it's understandable to think that way. But if you have, you realize the immense pressure, self-doubt, and scrutiny you'd have even making a bold, well researched, attempt to prove someone like knuth wrong even if it was obvious he was. I recall several instances in mathematics where this happened and it bordered on starting a war between mathematicians.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#106

I saw a tweet by a researcher who sent her work for peer review and got feedback that she should read more works of xxxxx and revise her work. Great feedback, she thought, because She was xxxxx.

Sometimes the peer review is the one asking for citing him/hers work, not anybody else's...

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#107
post #33

Earlier quoted context omitted.

Peer review is not supposed to be a Twitter popularity contest so these numbers are indeed astonishing.

Right, nothing is really. The parent is saying that this is a common human flaw and exists everywhere

Yes, quite. And that the blind spot is characteristic of economistic thinking.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#108
post #47
post #42

Earlier quoted context omitted.

That's usually a consequence of too much nepotism, or some super expensive piece of equipment. Double blind helps the former, though not the latter. In a part of my field that is capital intensive, some well funded newcomers have recently invested a lot, and "broke in", while some incumbent testbeds went away; the "expensive equipment" problem is usually temporary if the field is expanding.

It's just that experiments in astroparticle physics are expensive and have lots of people on them. Like, how is the IceCube Collaboration supposed to write an anonymous paper? Even the most cursory description of the detector would give it away...

Well, maybe there will be many IceCube's later? (Probably not, but I'm routinely shocked when I learn about new giant testbeds in my field...)

Alternatively, when IceCube 2 comes out, the old IceCube crowd might be focusing on other stuff, and not paying attention to the IceCube 2 politics. That makes them great peer reviewers (no horse in the race, but knowledgeable).

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#109
post #7

How is double blind not the norm? there is just too much room for bias

Past performance is a reasonable indicator in a lot of situations. I've had this discussion in the context of conference submittals. While there are lots of reasons to be inclusive of new presenters, the reality is that I also want to be aware as a conference committee member of applicants who have been very popular in the past.

I've found the opposite. I don't want to hear version 10 of a paper from 2005.

Besides, invited speakers are a thing for a reason.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#110

Very common for anyone who has ever spent time in academia. When I was still in grad school I noticed a trend in some major mathematical and CS journals where a lot of time it would be the same N people writing articles. This was driven home when my advisor told me "without such and such's name this paper has no chance of making it past review". Non-sense. However, for all the people that say "trust the science" this…

First of all, Knuth would never publish something "garbage" because his reputation would be on the line for everyone to read (forever).

Your response perfectly illustrates the point OP was making
Post reply on HN