Live data from Hacker News

It’s time to allow researchers to submit manuscripts to multiple journals

nature.com

61–70 of 126 posts

Re: It’s time to allow researchers to submit manuscripts to multiple journals

#61

I'd rather do away with the whole publish or perish thing. One half-baked idea: For academic hiring, only ever judge the quality of a candidate's research based on their five best papers (as they themselves have nominated), then there is no pressure to publish anything that doesn't break into the top five.

And then someone publishes five good papers and does nothing* for the rest of their life.

I mean, the idea looks nice, but there will always be people trying to game the system.

But don't get me wrong – I still like your idea, I just think it would need some refining (as you yourself admit, ofc).

* As in, no research.

Re: It’s time to allow researchers to submit manuscripts to multiple journals

#62

If the peer reivew is turned into a properly paid work through a platform, many researchers from under-developed countries will surely join to become reveiwers. This will bridge the demand-supply gap and and make the publishing faster. https://www.thelancet.com/journals/lancet/article/PIIS0140-6...

Who wants to pay for something that used to be free?

Re: It’s time to allow researchers to submit manuscripts to multiple journals

#63

I'd rather do away with the whole publish or perish thing. One half-baked idea: For academic hiring, only ever judge the quality of a candidate's research based on their five best papers (as they themselves have nominated), then there is no pressure to publish anything that doesn't break into the top five.

And then someone publishes five good papers and does nothing* for the rest of their life. I mean, the idea looks nice, but there will always be people trying to game the system. But don't get me wrong – I still like your idea, I just think it would need some refining (as you yourself admit, ofc). * As in, no research.

[dead]

Re: It’s time to allow researchers to submit manuscripts to multiple journals

#64
He mentions that there is a conflict of interest with recommending peer reviewers. While I agree this can be abused, I've often run into cargo cult science in AI when publishing something that is valid, novel, and in my opinion advances the field, because it is not aligned with how past work defined the problem when submitting to conferences where I cannot recommend more senior scientists as reviewers. Recommending people could help a lot to address this.

For example, in continual deep learning people often use time datasets in which they use a small amount of memory and incrementally learn classes and the algorithms cannot work for other incremental learning distributions. It's been very hard to publish work that instead works well for arbitrary multiple distributions, eliminates the memory constraint (which doesn't matter in the real world mostly), and shows it scales to real datasets. We have been able to get huge reductions in training time with no loss in predictive ability, but can't seem to get any of these papers published because the community says it is too unorthodox. It is far more efficient than periodically retraining as done in industry, which is what industry folks always tell me is the application they want from continual learning.

The confusing thing is that when I give talks or serve on panels I always have folks thank me and tell me they think this is the right direction and it was inspiring.

In my field the review system is way overtaxed with too many inexperienced people who struggle with having a broad perspective, so I think submitting to more venues would probably make things worse.

Re: It’s time to allow researchers to submit manuscripts to multiple journals

#65

If the peer reivew is turned into a properly paid work through a platform, many researchers from under-developed countries will surely join to become reveiwers. This will bridge the demand-supply gap and and make the publishing faster. https://www.thelancet.com/journals/lancet/article/PIIS0140-6...

If peer review becomes paid work, most journals will probably outsource reviewing to developing countries. Journals (effectively universities and research funders) just can't afford the consulting fees of first-world academics.

Re: It’s time to allow researchers to submit manuscripts to multiple journals

#66
It is time to burn traditional academic publishing to the ground and rebuild it from the ground up, or else let's rebel against the government, give Alexandra Elbakyan citizenship and put her in charge of the Department of Energy to make all computer science research open domain for all the world to see. Requesting $48 for to read a paper funded by your taxes before you even know it is useful to your work will result in deportation or a firing squad, whichever a panel of independent reviewers decides.

A less spirited method is to let traditional publishing slowly die by adding review features to highly viable alternatives like arXiv, PubMed and others which will then replace it. I hate academic publishers, they are vultures that serve no purpose. I hope they all go out of business or somehow pay big :)

Re: It’s time to allow researchers to submit manuscripts to multiple journals

#67
I've often wondered if it might be possible to set up an 'auction' system for papers. Post on arXiv and then have journals 'bid' for publication - saying how much they would charge in publication fees, review turnaround etc. Authors can then choose the journal they prefer. The advantage would be that the initial stage of 'shopping around' would be eliminated (sending a paper to a journal and being rejected without review) and there would be incentive for journals to reduce publication fees. Just a thought...

Re: It’s time to allow researchers to submit manuscripts to multiple journals

#68
post #7

Computer Science's conventions solve these problems (although CS certainly has other problems): * papers are generally posted to arxiv.org immediately on being finished, so everyone can access them * conferences have fixed deadlines and relatively short, fixed/enforced review cycles

That's a good start. It's certainly a major improvement from when I still published regularly (until about 15 years ago).

One thing that struck me as a researcher back in the day was that it was much easier to get people reading things that I put on my blog than it was getting scientists to read my papers. Basically as a researcher, you are basically engaging in 17th century SEO by writing papers and getting your peers to pay attention to this. We use email these days but otherwise the process hasn't changed a lot.

This is weird considering we now have a lot more tools. Imagine if hacker news worked that way. It wouldn't work. The best kind of endorsement for a paper is not random anonymous peer reviewers giving their thumbs up or down as to the inclusion of a digital only journal that nobody ever reads cover to cover. Instead it is other researchers citing your work. This is in fact part of how most academic performance is measured. The goal is to get your peers being aware of the existence of your work, get them to spend time reading or learning about it, and then getting them to engage with it by citing, criticizing, or adding to it.

The whole business of a journal publication is just SEO. You get some renowned journal to include you and maybe people will bother reading it and maybe refer your work if they like it. A citation is just a glorified like. The real goal is to get people to read and "like" your work. It's also self re-enforcing: the more people cite your work, the more people will read it and thus cite it. But it's a super inefficient process. From idea to citations happening can take years.

Conferences and workshops are where scientists meet up and discuss their work. That's where your peers are. I always enjoyed the smaller workshops. Get some smart people in a room and beautiful things happen. The academic world is basically a social network. With likes and everything. But minus all the modern tools that make other social networks work so well. There's some room for improvement.

Re: It’s time to allow researchers to submit manuscripts to multiple journals

#69

Peer review is a joke. Peer reviewers don’t look at your data and the programs you used to analyze it. They don’t look at your experimental apparatus, they don’t repeat your experiment, they don’t talk to the subjects you interviewed, at best they can spot obvious “red flags”. (Of course, in 2023 you should be able to publish your data and all your software with the paper.)

I wouldn’t say it’s a joke, rather it’s not perfect.

When papers are reviewed, there are going to be a finite number of spots in the journal or conference to be assigned competitively. In good places, the reviewers catch issues in the papers and it won’t be easy to pass them.

Without peer review, a PhD student requesting graduation or a candidate applying for a faculty position would claim they have done major work, and there is no way to filter out the noise.

Re: It’s time to allow researchers to submit manuscripts to multiple journals

#70

I'd rather do away with the whole publish or perish thing. One half-baked idea: For academic hiring, only ever judge the quality of a candidate's research based on their five best papers (as they themselves have nominated), then there is no pressure to publish anything that doesn't break into the top five.

And then someone publishes five good papers and does nothing* for the rest of their life. I mean, the idea looks nice, but there will always be people trying to game the system. But don't get me wrong – I still like your idea, I just think it would need some refining (as you yourself admit, ofc). * As in, no research.

I don’t see a problem with someone publishing 5 great papers early in their career and nothing else. I mean what were the meaningful publications from Oppenheimer in his later years? Or Einstein? Publishing should be done when it matters, not when it’s needed for a salary increase.
Post reply on HN