Why isn't preprint review being adopted?

> I wouldn't say peer review is useless, I think it did improve papers I wrote a little

Let's get back to basics here. What is the whole point of doing things scientifically in the first place? I.e. what is so special about the combination of 1) communicable descriptions of experiments, which can be re-run by anybody, and 2) the performing of said experiments and reporting the results, and 3) having others review and reproduce the results?

Why are those 3 practices soooo special? Its because they are designed to either a) persuade everybody to agreement with you that your results are valid, or b) persuade you that you are wrong.

I.e.. the scientific method is uniquely designed for the purpose of compelling agreement among peers. Its the 2nd best method we have of reaching consensus (the first best being the methods of proof used in mathematics. )

This is important because even if you make a new gee-whiz discovery, its useless unless other people agree that it is real. And unlike politics, or a beauty contest, or an aristocracy, it doesn't matter who you are, if your results are observable and reproducible, other scientists will agree with you, and your discovery has some chance of being generally useful. And that's why they are anonymous--its supposed to cut through the biases and prejudices which, alas, sometimes prevent us from coming to consensus.

The point of peer review is to get a read on whether or not the author of the paper has justified their claims in enough detail that the results are likely to compile agreement. Yeah, we see if 3 or 4 people can agree on it, before asking everybody to agree on it. Whoda thunk?

Point being, peer review isn't to help you improve your paper.

> reviewers do not put enough effort into to reliably catch validity problems.

Yes. This. Any why don't they? Because there's only so much work you can expect somebody to do for free. Lets pay them.

Like I said, it's not clear to me what exactly established journals have been doing "better" historically than a preprint server could. You say they are better at "being a journal" -- OK. They are established, well connected (to science communities, industry, journalism, funding agencies, etc.) and have been maintaining a reputation for a long time, usually much longer than preprint servers exist. That's basically prestige, which isn't nothing (I didn't claim prestige is nothing). However, this doesn't demonstrate journals do anything relevant to the advancement of science fundamentally better right now.

What I "have" is that

1. It's not obvious that a journal is fundamentally better at organizing unpaid voluntary reviewers compared to a preprint server.

2. Scientific publishing has insanely high profit margins. How come? My theory is that they are selling prestige first and foremost, i.e., a luxury good, (to scientists, universities and funding agencies simultaneously) and purchasing decisions there are made by people who are spending public money, not their own. Both of these points (luxury good, public spending) seem like strong contributors to high margins. The public is paying for the research and for access to articles, while journals nowadays on first glance seem to only provide a little bit of web hosting, a little bit of reviewer coordination and a little bit of manuscript editing.

3. It's not obvious that the submission and peer review systems we have now (in journals) is worth the time and effort. The role of peer review is misrepresented in journalism and the expectations are not met. If one could separate publication ("preprint") on one side, and, on the other side, review and "being featured by important outlets or institutions", authors could save a lot of time time (that could be used for more research). Others would have access to interesting results earlier and be able to build on top of them. Next, in a separate process some institution could select important works, scrutinize and review them, perhaps paying experts to do so, and perhaps replicate where appropriate.

The issue with this is that academics need the prestige provided by journals for career advancement, universities need the prestige to justify their spending to funding agencies and politicians, and funding agencies likewise need the prestige to justify their spending to politicians. The "replication crisis" and the like indicate that this prestige is overvalued. The hope is, economically speaking, that the market for "academic prestige" can either be disrupted, or the price the public has to pay can be lowered "through competition". It's interesting what that might look like. Preprint servers, open data and more direct science communication seem like steps in the right direction.

Do you have a URL/citation for that? I'd like to use this quote in a paper.

Sure, but I’m poking at the bad leverage you can also wield.

1) you can create undue schedule pressure that results in a cursory review that may not catch the more nuanced problems in your investigation.

2) you can be more belligerent about not sharing data. If they want to get, they won’t argue.

3) you can pay for reviewers who you know will give a positive review. Without guards against this, it’s almost a certainty that the glut of PhDs will result in some treating it like a scam de hustle where it’s more about the economics than the science.

Some consultants are well known to play the game where they tell clients what they want to hear rather than what they need to hear. I don’t think consultancy is a good model for this.

I didn't say that. It needs something. Handwaving about an emergent community isn't useful - moving from todays system to something else needs something concrete.