Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why isn't preprint review being adopted?

theroadgoeson.com

Post reply on HN
Print view View original

51–60 of 169 posts

Re: Why isn't preprint review being adopted?

#51

Post by BlueTemplar »

freedomben wrote: post #48
zer00eyz wrote: post #23

Earlier quoted context omitted.

There is a pretty simple way to change all of that. Academic standards: You are not longer allowed to site a non reproduced paper in yours. Citations matter as much as the print, put the hurdle there and all of a sudden things will change real quick.

> You are not longer allowed to site a non reproduced paper in yours. I fully agree that in an ideal world, that would be the case. But some reproductions (especially now with machine learning) could cost millions of dollars and years to do. I don't think that's a reasonable or feasible thing to require.

Well, then, at least there should be pressure around having to mention this : a non-replicated study (by an independent group) is after all inherently suspect.

Re: Why isn't preprint review being adopted?

#52

Post by salty_biscuits »

adastra22 wrote: post #38

PREPUBLICATION REVIEW IS BAD! STOP TRYING TO REINTRODUCE IT. Sorry for the all caps. Publishing papers without “peer review” isn’t some radical new concept—it’s how all scientific fields operated prior to ca. 1970. That’s about when the pace of article writing outstripped available pages in journals and this system of pre-publication review was adopted and formalized. For the first 300 years of science you published…

The whole process comes from a time when publishing was expensive, should be cheap as chips now. The system needs a rethink so "quality" can somehow bubble up to the surface given mass publication is so simple.

Re: Why isn't preprint review being adopted?

#54

Post by BlueTemplar »

adastra22 wrote: post #38

PREPUBLICATION REVIEW IS BAD! STOP TRYING TO REINTRODUCE IT. Sorry for the all caps. Publishing papers without “peer review” isn’t some radical new concept—it’s how all scientific fields operated prior to ca. 1970. That’s about when the pace of article writing outstripped available pages in journals and this system of pre-publication review was adopted and formalized. For the first 300 years of science you published…

Also, the author IMHO failed to clearly explain how "preprint review" wasn't a contradiction in terms (though they do seem to gesture towards the main issue being commercialization of journals, in the first post).

In the same vein, the positive mention of Github and "open source platform" (another contradiction in terms) were at first red flags in the third article, but at least they then mentioned the threat of corporate takeover.

Re: Why isn't preprint review being adopted?

#55

Post by cachemoi »

The solution to the dated model exists, it's git/github. I'm trying to build a "git journal" (essentially a github org where projects/research gets published paired with a substack newsletter), details here [0]

Let me know if you have a project you'd like to get on there! Here's what it looks like, a paper on directed evolution [1]

[0] https://versioned.science/ [1] https://github.com/versioned-science/DNA_polymerase_directed...

Re: Why isn't preprint review being adopted?

#57

Post by SkyPuncher »

adastra22 wrote: post #38

PREPUBLICATION REVIEW IS BAD! STOP TRYING TO REINTRODUCE IT. Sorry for the all caps. Publishing papers without “peer review” isn’t some radical new concept—it’s how all scientific fields operated prior to ca. 1970. That’s about when the pace of article writing outstripped available pages in journals and this system of pre-publication review was adopted and formalized. For the first 300 years of science you published…

I've found the entire peer review concept in academia to be extremely odd. I'm not entirely sure what problem it solves. It seems like you have one of two types of people reading these articles:

* People who are already specialists/familiar enough with concepts. They'll either call BS upfront or run experimentation themselves to validate results.

* People who aren't specialists and will need to corroborate evidence against other sources.

My entire life as a software engineer has been built blogs, forums, and discussion from "random" people doing exactly the above.

Re: Why isn't preprint review being adopted?

#58

Post by adastra22 »

freedomben wrote: post #43
adastra22 wrote: post #38

PREPUBLICATION REVIEW IS BAD! STOP TRYING TO REINTRODUCE IT. Sorry for the all caps. Publishing papers without “peer review” isn’t some radical new concept—it’s how all scientific fields operated prior to ca. 1970. That’s about when the pace of article writing outstripped available pages in journals and this system of pre-publication review was adopted and formalized. For the first 300 years of science you published…

> The idea of having your intellectual competitors review your work potentially sabotage your publication prospects as a standard process is a relatively recent addition. And one that has not been shown to actually be effective. If this is true (and I'm not doubting you, just acknowledging that I'm taking your word for it) then why abandon the entire system? Why not just roll it back to the state before we added the…

The prior state was a situation of no pre-publication review other than the editorial staff of the journal. We should go back to that, yes. By disbanding entirely the “peer review” system that currently exists.

Re: Why isn't preprint review being adopted?

#60

Post by YeGoblynQueenne »

ccppurcell wrote: post #44

Peer review is pretty unpopular round these parts. In mathematics/TCS I've had mostly good experiences. Actually most of the time the review process improved my papers. Clearly something is rotten about the way peer review is implemented in the empirical sciences. But think of all those high profile retractions you read about these days. Usually that comes about by a sort of post hoc peer review, not by anything rese…

Not to be mean to the HN community but at least a substantial minority of people who complain about peer review on here have no experience of peer review, even in applied CS and AI and machine learning, which are the hot topics today. But they've read that peer review is broken and, by god, they'll let the world know! For science!

I publish in machine learning and my experience is the same as yours: reviews have mainly helped me to improve my papers. Though to be fair this is mainly the case in journals; in conferences it's true that reviewers will often look for reasons to reject and don't try to be constructive (I always do; I still find it very hard to reject).

This is the result of the field of AI research having experienced a huge explosion of interest, and therefore submissions, in the last few years, so that all the conferences are creaking under the strain. Most of the new entrants are also junior researchers without too much experience- and that is true for both authors and reviewers (who are only invited to review after they publish in a venue). So the conferences are a bit of a mess at the moment, and the quality of the papers that get submitted and published, overall low.

But that's not because "peer review is broken", it's because too many people start a PhD in machine learning thinking they'll get immediately hired by Google, or OpenAI I guess. That too shall pass, and then things will calm down.

Post reply on HN