Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

It’s time to allow researchers to submit manuscripts to multiple journals

nature.com

Post reply on HN
Print view View original

21–30 of 126 posts

Re: It’s time to allow researchers to submit manuscripts to multiple journals

#21

Post by radus »

nextos wrote: post #16
ssivark wrote: post #6

The best way to allow multiple submissions while amortizing the review work is to make reviews (and rebuttals) public, and overlay them on submissions made public on some preprint server. This establishes priority (if credit is a concern), can be made blind / double-blind if so desired, and also makes public the reviews (which are as much a public service as writing research papers). Which editorial boards “accept” t…

With a few minor changes, eLife is a non-profit journal that has that kind of peer review process: https://elifesciences.org/about/peer-review It is quite well regarded now, they publish excellent research and the whole process is crystal clear.

One of my favorite journals. Perhaps of interest to HNers is their fairly active GitHub: https://github.com/elifesciences, and my favorite project of theirs, Lens: https://github.com/elifesciences/lens, it's a paper viewer that actually makes reading papers easier (there's loads of these out there but most are crap and inferior to a PDF).

Re: It’s time to allow researchers to submit manuscripts to multiple journals

#22

Post by fsckboy »

PaulHoule wrote: post #2

Peer review is a joke. Peer reviewers don’t look at your data and the programs you used to analyze it. They don’t look at your experimental apparatus, they don’t repeat your experiment, they don’t talk to the subjects you interviewed, at best they can spot obvious “red flags”. (Of course, in 2023 you should be able to publish your data and all your software with the paper.)

> Peer review is a joke. Peer reviewers don’t look at your data and the programs you used to analyze it. They don’t look at your experimental apparatus, they don’t repeat your experiment, they don’t talk to the subjects you interviewed, at best they can spot obvious “red flags”.

if those were the worst problems with peer review, we'd be in a much better place. Your peer reviewers are frequently higher status scientists working (competing) in the same research area you are trying to publish in. Generally, they do not want their own work outshined or overthrown.

Re: It’s time to allow researchers to submit manuscripts to multiple journals

#24

Post by aydyn »

PaulHoule wrote: post #2

Peer review is a joke. Peer reviewers don’t look at your data and the programs you used to analyze it. They don’t look at your experimental apparatus, they don’t repeat your experiment, they don’t talk to the subjects you interviewed, at best they can spot obvious “red flags”. (Of course, in 2023 you should be able to publish your data and all your software with the paper.)

Idk about you guys but the only reason I do peer review is to reject competitors and enemies.

If I really hate them, I give them a "Major Revision" aka a laundry list of expensive follow-up experiments and then reject them on the 2nd or 3rd round after a few months.

There's actually zero benefit to earnestly engaging in peer review.

Re: It’s time to allow researchers to submit manuscripts to multiple journals

#25

Post by bluenose69 »

The author suggests that "The fear that multiple submissions would overwhelm the peer-review system lacks empirical evidence". Maybe it won't "overwhelm" it, but it will certainly add to the reviewing workload. Simply stated, if authors submit to N journals and each asks for 2 reviewers, that's 2N units of work (assuming they can get the reviewers), compared to 2 units of work.

But it may be worse than that, actually. I will not be much inclined to bother reviewing, if I know that the authors might pull their manuscript if another journal gives a green light quicker than the journal for which I have been asked to review.

The solution to a slow reviewing process is not to ask reviewers to do more of this unrewarded work.

Re: It’s time to allow researchers to submit manuscripts to multiple journals

#26

Post by sxg »

radus wrote: post #17
PaulHoule wrote: post #2

Peer review is a joke. Peer reviewers don’t look at your data and the programs you used to analyze it. They don’t look at your experimental apparatus, they don’t repeat your experiment, they don’t talk to the subjects you interviewed, at best they can spot obvious “red flags”. (Of course, in 2023 you should be able to publish your data and all your software with the paper.)

Some of the things that peer reviewers do, in my experience, in biology: - question whether or not the conclusions you are making are supported by the data you are presenting - ask for additional experiments - evaluate whether or not your research is sufficiently novel and properly contextualized - spotting obvious red flags - you seem to discount this, but it's quite valuable In my experience, the process of peer re…

I also work in biological sciences research, but I'm more skeptical of peer review than you appear to be. My main criticism is that peer review is an n=2 process. Why not publish an unreviewed pre-print in bioRxiv and explicitly solicit constructive, public feedback directly on the pre-print on bioRxiv? I envision something similar to GitHub where users can open issues and have nuanced discussions about the work. The authors can address these issues by replying to users and updating the data and/or manuscript while bioRxiv logs the change history. Journals can then select sufficiently mature manuscripts on bioRxiv and invite the authors to publish.

This would massively increase the number of people that review a manuscript while also shortening the feedback cycle. The papers I've published have typically been in the peer review process for months to years with just a handful of feedback cycles of sometimes dubious utility. This can be improved!

Edit: I forgot to mention the issue of politics in peer review! If you're in a relatively small field, most of the big researchers all know each other, so peer review isn't truly blinded in practice. Junior researchers are also pressured into acquiescing to the peer reviewers rather than having an actual scientific debate (speaking from experience).

Re: It’s time to allow researchers to submit manuscripts to multiple journals

#27

Post by aydyn »

0cf8612b2e1e wrote: post #8
PaulHoule wrote: post #2

Peer review is a joke. Peer reviewers don’t look at your data and the programs you used to analyze it. They don’t look at your experimental apparatus, they don’t repeat your experiment, they don’t talk to the subjects you interviewed, at best they can spot obvious “red flags”. (Of course, in 2023 you should be able to publish your data and all your software with the paper.)

Fraud is considered rare, and trust is fundamental. In which case, you choose to believe what they said they did and interrogate of what they said they did is reasonable. Nobody has the budget, time, and sometimes magical fingers required to reproduce every submission. You can disagree with this approach, but then there needs to be huge budgets set aside for reproduction.

> Fraud is considered rare, and trust is fundamental.

This is a nice sentiment but demonstrably false.

Fraud is common in academia and everyone knows it. A large part of academic science is a grift for funding. Is not "Trust" that is fundamental, is tit-for-tat.

Re: It’s time to allow researchers to submit manuscripts to multiple journals

#28

Post by BeetleB »

aydyn wrote: post #24
PaulHoule wrote: post #2

Peer review is a joke. Peer reviewers don’t look at your data and the programs you used to analyze it. They don’t look at your experimental apparatus, they don’t repeat your experiment, they don’t talk to the subjects you interviewed, at best they can spot obvious “red flags”. (Of course, in 2023 you should be able to publish your data and all your software with the paper.)

Idk about you guys but the only reason I do peer review is to reject competitors and enemies. If I really hate them, I give them a "Major Revision" aka a laundry list of expensive follow-up experiments and then reject them on the 2nd or 3rd round after a few months. There's actually zero benefit to earnestly engaging in peer review.

You are an exemplar of all that is wrong in academia, but I upvoted you because there are so many like you.

(I know it from personal experience).

Personally, I decided to leave and make a more honest living. It seems you chose not to.

Re: It’s time to allow researchers to submit manuscripts to multiple journals

#29

Post by impendia »

liliumregale wrote: post #15
bo1024 wrote: post #7

Computer Science's conventions solve these problems (although CS certainly has other problems): * papers are generally posted to arxiv.org immediately on being finished, so everyone can access them * conferences have fixed deadlines and relatively short, fixed/enforced review cycles

Let's distinguish between papers and preprints, please. arXiv has contributed to a blurring of the distinction. The arXiv preprints are useful but should always be taken with a grain of salt. There is nearly no filtering done on things uploaded to arXiv. Everyone accessing someone's uncritically reviewed work is a bittersweet gift.

In mathematics, at least, papers and preprints are indeed widely considered to be the same thing. In practice, for people working in the field, they are.

Math papers tend to be highly technical, read by other specialists in the field. When it comes for correctness -- whether or not I should take a paper with a grain of salt -- the authors' reputation counts for much more than the journal's. And in case of student authors, who are just beginning to publish, the advisor is implicitly staking their reputation on the work as well.

There are also preprints on the arXiv, written by people unknown in the community, claiming to prove the Riemann Hypothesis or some such. These aren't taken seriously by anyone.

An outsider might not be able to tell which preprints can be considered equivalent to papers, but such people are not likely to be seriously reading math research in the first place.

Re: It’s time to allow researchers to submit manuscripts to multiple journals

#30

Post by dbingham »

sxg wrote: post #26
radus wrote: post #17

Earlier quoted context omitted.

Some of the things that peer reviewers do, in my experience, in biology: - question whether or not the conclusions you are making are supported by the data you are presenting - ask for additional experiments - evaluate whether or not your research is sufficiently novel and properly contextualized - spotting obvious red flags - you seem to discount this, but it's quite valuable In my experience, the process of peer re…

I also work in biological sciences research, but I'm more skeptical of peer review than you appear to be. My main criticism is that peer review is an n=2 process. Why not publish an unreviewed pre-print in bioRxiv and explicitly solicit constructive, public feedback directly on the pre-print on bioRxiv? I envision something similar to GitHub where users can open issues and have nuanced discussions about the work. The…

As it happens, I'm building "Github for Journals".

I pivoted away from attempting a crowd source review approach with a reputation system to trying to support journals in going Diamond Open Access.

But the platform I've built supports co-author collaboration, preprints and preprint review, journal publishing flows, and post publication review - all in a continuous flow that utilizes an interface drawing from Github PRs and Google Docs.

You can submit a paper, collect feedback from co-authors, then submit it as a preprint and collect preprint feedback, then submit to a journal and run the journal review process, then collect feedback on the final published paper. And you can manage multiple versions of the paper, collecting review rounds on each version, through that whole process.

It's in alpha, I'm pushing really hard with a short runway to get the journal flows to usable beta while trying to raise seed funding... the catch being I feel very strongly that it needs to be non-profit, so seed funding here is grants and donations.

I'm looking for journal editors who want to participate in UX research. I'm also interested in talking to folks who run preprint servers to see if they'd have any interest in using the platform. If you (being any reader) know any, or have leads for funding, reach out: dbingham@theroadgoeson.com

Post reply on HN