It’s time to allow researchers to submit manuscripts to multiple journals
1–10 of 126 posts
Re: It’s time to allow researchers to submit manuscripts to multiple journals
#2Peer review is a joke. Peer reviewers don’t look at your data and the programs you used to analyze it. They don’t look at your experimental apparatus, they don’t repeat your experiment, they don’t talk to the subjects you interviewed, at best they can spot obvious “red flags”.
(Of course, in 2023 you should be able to publish your data and all your software with the paper.)
Re: It’s time to allow researchers to submit manuscripts to multiple journals
#3"The fear that multiple submissions would overwhelm the peer-review system lacks empirical evidence and is outweighed by the burden placed on researchers."
Actually, here's a paper showing that the peer review is already overstretched:
Re: It’s time to allow researchers to submit manuscripts to multiple journals
#4Peer review is a joke. Peer reviewers don’t look at your data and the programs you used to analyze it. They don’t look at your experimental apparatus, they don’t repeat your experiment, they don’t talk to the subjects you interviewed, at best they can spot obvious “red flags”. (Of course, in 2023 you should be able to publish your data and all your software with the paper.)
I partially agree, and I can enumerate other issues with peer review that you have not listed, but it is worthwhile to point out some of the positive features of the peer review concept:
- Peer review in reputable non-profit journals actually provides constructive suggestions that make papers and research itself better. APS's PRX and PRL, as well as Quantum are journals where I have seen these idealistic positive effects;
- Filtering out the obvious red flags is pretty valuable even if boring;
- Thanks to people who care about the "ideal" of peer review we now have the infrastructure necessary to make reproducability much easier: mandatory data and code sharing on archival services, open (even crowdsourced) peer review, immediate feedback, etc.
Re: It’s time to allow researchers to submit manuscripts to multiple journals
#5Peer review is a joke. Peer reviewers don’t look at your data and the programs you used to analyze it. They don’t look at your experimental apparatus, they don’t repeat your experiment, they don’t talk to the subjects you interviewed, at best they can spot obvious “red flags”. (Of course, in 2023 you should be able to publish your data and all your software with the paper.)
It still is a quite useful filter, as without it most fields would be even more overwhelmed. As a reviewer, have you seen what garbage gets submitted sometimes? There are incentives to attempt to get garbage published, so throwing out a significant part of submissions does add quite a lot of value to readers, so that they get at a somewhat curated list of papers from that journal or conference.
And while all you say is true, it's probably the most we can get for free in a reasonable amount of time; requiring an independent lab to repeat an experiment would generally be far more delay and cost than we'd accept, other researchers do generally want to see the outcome as soon as the first experiment is documented; and there are people doing great research which won't bother to submit if they'd have to pay for the replication - it's generally the bad research that has motivation to spend more money for a publication. The general public might want to wait for extra confirmation, but they're not the target audience of research papers, those are intended as communication by researchers for researchers. And also quite a few media outlets would disagree and probably prefer grabbing up hot rumors even earlier, even if they turn out to be false afterwards.
Re: It’s time to allow researchers to submit manuscripts to multiple journals
#6The best way to allow multiple submissions while amortizing the review work is to make reviews (and rebuttals) public, and overlay them on submissions made public on some preprint server.
This establishes priority (if credit is a concern), can be made blind / double-blind if so desired, and also makes public the reviews (which are as much a public service as writing research papers). Which editorial boards “accept” the paper for publication is then simply a matter of collecting endorsement tags on the submission.
Re: It’s time to allow researchers to submit manuscripts to multiple journals
#7Computer Science's conventions solve these problems (although CS certainly has other problems):
* papers are generally posted to arxiv.org immediately on being finished, so everyone can access them
* conferences have fixed deadlines and relatively short, fixed/enforced review cyclesRe: It’s time to allow researchers to submit manuscripts to multiple journals
#8Peer review is a joke. Peer reviewers don’t look at your data and the programs you used to analyze it. They don’t look at your experimental apparatus, they don’t repeat your experiment, they don’t talk to the subjects you interviewed, at best they can spot obvious “red flags”. (Of course, in 2023 you should be able to publish your data and all your software with the paper.)
Fraud is considered rare, and trust is fundamental. In which case, you choose to believe what they said they did and interrogate of what they said they did is reasonable. Nobody has the budget, time, and sometimes magical fingers required to reproduce every submission.
You can disagree with this approach, but then there needs to be huge budgets set aside for reproduction.
Re: It’s time to allow researchers to submit manuscripts to multiple journals
#9Peer review is a joke. Peer reviewers don’t look at your data and the programs you used to analyze it. They don’t look at your experimental apparatus, they don’t repeat your experiment, they don’t talk to the subjects you interviewed, at best they can spot obvious “red flags”. (Of course, in 2023 you should be able to publish your data and all your software with the paper.)
Fraud is considered rare, and trust is fundamental. In which case, you choose to believe what they said they did and interrogate of what they said they did is reasonable. Nobody has the budget, time, and sometimes magical fingers required to reproduce every submission. You can disagree with this approach, but then there needs to be huge budgets set aside for reproduction.
Fraud is considered rare, but maybe not actually that rare; hence the replication crisis.
Re: It’s time to allow researchers to submit manuscripts to multiple journals
#10Peer review is a joke. Peer reviewers don’t look at your data and the programs you used to analyze it. They don’t look at your experimental apparatus, they don’t repeat your experiment, they don’t talk to the subjects you interviewed, at best they can spot obvious “red flags”. (Of course, in 2023 you should be able to publish your data and all your software with the paper.)
It still is a quite useful filter, as without it most fields would be even more overwhelmed. As a reviewer, have you seen what garbage gets submitted sometimes? There are incentives to attempt to get garbage published, so throwing out a significant part of submissions does add quite a lot of value to readers, so that they get at a somewhat curated list of papers from that journal or conference. And while all you say…
All of what you wrote is true too, but it’s also the hollowed out support beam at bottom of “evidence-based everything” culture, which has taken over almost everything.
The truth is that good science is slow and that most “evidence-based” practices are referring to a huge, nebulous cloud of bad results and weak suggestions rather than the evidence that supposedly gives them authority over traditional or intuitive practices.
Scientists participate on “Little Science” and the responsible ones often maintain the perspective that you’re describing here.
But modern society has built itself around the institution of “Big Science” which is structurally forced to assert truths before they can responsibly be defended.
It’s way bigger than the general public being curious or the media wanting to get eyeballs — it’s everything going on in government, economics, medicine, psychology, agriculture, etc etc etc
It’s a house of cards and you’ve just summarized what the core problem is.