Live data from Hacker News

Authors’ names have ‘astonishing’ influence on peer reviewers: study

nature.com

191–200 of 319 posts

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#191

Earlier quoted context omitted.

The problem with that is, at least in my experience, that you read such things once from a reputable source and never check it again.

When I was studying undergrad psych, Psychology Today was falling into disrepute. Are they still a reputable source, today?

Psychology Today is not great in general, but the article I linked to is written by a social psychology PhD (Lee Jussim).

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#192

Earlier quoted context omitted.

You trust science because it has the highest probability of being accurate. What's the alternative? Most people aren't capable of analysis scietific research to see if it's correct. What makes something political? If new research shows that X product is dangerous and republicans, who that company donates to, come out and disagree with the research they just made it political. Now, someone like you, can just say "This…

Jesus Christ, the point of science is to REMOVE trust - not be the least worst option. Science that cannot exist without trust is faith.

https://lemire.me/blog/2020/07/12/science-is-the-belief-in-t...

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#193

That’s what we want in society right? Perks for doing hard work and then being recognized for it. That’s what “street cred” is. Maintaining a reputation is hard work and the fast pass and acknowledgment from the broader society is your reward. As in this person has more than proved himself, let’s not vet him as much.

Linus Pauling won _two Nobel prizes_. He also, particularly in later life, got into weird quackery.

You don’t want to go on reputation.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#194

Earlier quoted context omitted.

Whoever told you that about your cites is probably right -- you shouldn't cite yourself while calling out that you are citing yourself. You should cite yourself the same you that site any other work. The second part is probably unavoidable. If you're working on something super specialized when all your reviewers are the six other people who work on it then sure, it'll be unblinded. Not much we can do there sadly.

I disagree, the style to write as a "neutral" observer (often writing in passive voice) is frowned upon nowadays for good reason. Part of that is also that the information if the authors wrote a citation or not can be important to a reviewer. For example it is unfortunately quite common that authors publish results in a salami tactic to maximize the number of publications. There can be a significant difference in imp…

Like sibling I'd also like more information on this and can't find anything immediately compelling with a light search. Like this post suggests, obviously there's a use to hiding information, sometimes a bad reputation is undeserved or irrelevant to the work, sometimes there are subtle biases in play.

The reputation of the author and their behaviour wrt the citations used should be considered, I agree with you that it's important information. Maybe the reviewer of an individual paper shouldn't be considering that though, maybe that should primarily be considered in a second review stage or in the context of meta-reviews? Idk, but the spirit of using passive voice in the context of research makes more sense to me.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#195

Double-anonymous/blind reviewing is completely standard in most areas of computer science, e.g. networking (SIGCOMM, NSDI, IMC, HotNets, MobiCom, MobiSys), systems (OSDI, SOSP, USENIX ATC, HotOS), security (Usenix Security, S&P), machine learning (NeurIPS and ICML), graphics and HCI (SIGGRAPH and CHI), and at least some top-tier theory conferences (like FOCS). So I think most computer scientists would agree with the…

In my experience, at least in the field of Solid-state Physics, double-blind review is quite rare. Also, it would be challenging to implement, since it's often very clear who is the group that submitted a paper. The community is not so big, everyone know everyone, samples and techniques are quite peculiar for each group and a paper typically cite a lot of previous work, to avoid large repetitions. It's not uncommon to guess well the reviewers

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#196

Earlier quoted context omitted.

When I was studying undergrad psych, Psychology Today was falling into disrepute. Are they still a reputable source, today?

Psychology Today is not great in general, but the article I linked to is written by a social psychology PhD (Lee Jussim).

Ok, good to know, thanks!

Setting this aside for a deeper read, but it appears at first glance that the concerns revolve around statistical technique rather than methodological soundness?

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#197

Earlier quoted context omitted.

Whoever told you that about your cites is probably right -- you shouldn't cite yourself while calling out that you are citing yourself. You should cite yourself the same you that site any other work. The second part is probably unavoidable. If you're working on something super specialized when all your reviewers are the six other people who work on it then sure, it'll be unblinded. Not much we can do there sadly.

I disagree, the style to write as a "neutral" observer (often writing in passive voice) is frowned upon nowadays for good reason. Part of that is also that the information if the authors wrote a citation or not can be important to a reviewer. For example it is unfortunately quite common that authors publish results in a salami tactic to maximize the number of publications. There can be a significant difference in imp…

> For example it is unfortunately quite common that authors publish results in a salami tactic to maximize the number of publications

Why is this unfortunate? I'd argue that splitting results in multiple publications is a) Riskier for authors (higher chance of rejection) and b) More convenient for readers (each paper requires less mental load, being focused on a single aspect). So, even if there's a payoff for authors, it doesn't come for free.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#198

Earlier quoted context omitted.

Academics just like every other aspect of human life is full of politics and this includes peer reviews as well. Papers regularly get panned for variety of reasons that have nothing to do with actual content of the paper and maybe even worse sometimes because the reviewer happen to be working on something similar and want secure a bigger grant for themselves.

That's false. Trust the science.

"Trust the science" is an oxymoron. Please do substantiate why grandparent's claim is false. There is a massive literature on the shortcomings and, yes, even potential for malice, of current peer-review practices.

https://sigbed.org/2022/08/22/the-toxic-culture-of-rejection...

https://www.nature.com/nature-index/news-blog/research-misco...

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#199
post #111

Earlier quoted context omitted.

The data is objective but the quality of the research is subjective. For example if my sample size was 150 for some study that's subjective whether it was sufficient of size

Not true. There are objective statistical tests regarding sample sizes.

There are objective measures of statistical power, given an effect size a priori ahead of time you can estimate the power of a particular procedure. The trouble is that what a "reasonable" effect size may be is subjective and requires prior knowledge; post-hoc power calculations are widely regarded as misleading and conveying little additional information beyond a p-value.

Re: Authors’ names have ‘astonishing’ influence on peer reviewers: study

#200

Earlier quoted context omitted.

Whoever told you that about your cites is probably right -- you shouldn't cite yourself while calling out that you are citing yourself. You should cite yourself the same you that site any other work. The second part is probably unavoidable. If you're working on something super specialized when all your reviewers are the six other people who work on it then sure, it'll be unblinded. Not much we can do there sadly.

I disagree, the style to write as a "neutral" observer (often writing in passive voice) is frowned upon nowadays for good reason. Part of that is also that the information if the authors wrote a citation or not can be important to a reviewer. For example it is unfortunately quite common that authors publish results in a salami tactic to maximize the number of publications. There can be a significant difference in imp…

Writing about your own work in third person has nothing to do with using the passive voice. You can use the active voice just fine: "Papers X,Y,Z show that ..." works regardless of the identity of the authors of X,Y,Z So the style argument is wrong.

The other argument you have is also questionable. It doesn't matter at all the identity of the citer when using citations to argue that a topic of research is important. If you cite 20 papers and they are mostly from the very same author, it doesn't matter if you're the author: any reviewer will realize the claim is shaky -- or not, if all those papers happens to be actually outstanding.

Double blind is imperfect but miles better than single blind. And we shouldn't list made-up defects that don't stand to scrutiny to it.

Post reply on HN