Why isn't preprint review being adopted?

Your other comment didn't say who might be paying reviewers. Journals clearly won't (why should they, they have grrat profits in the current system and will fight tooth and nail to delay any changes whatever). Universities and even funding agencies cannot (conflict of interest).

> Is the search engine going to do a better job than peer review of presenting you the papers you actually want to read?

I do actually expect to see that happen.

In the parent comment to this thread, I talk about this. My proposal is that when a researcher writes up the grant to get their research funded, they should estimate how much it would cost to pay reviewers and replicators, and include those figures in the cost.

If the researcher gets the grant approved, then the funding agency will put the money for peer review/replication into escrow. When the research is finished and the investigators have written up a paper to describe their methods and results, the money in escrow is disbursed to the reviewers and replicators.

If reviewers agree its good, and if it replicates, then the paper is published. If not, well we just dodged a bullet.

> I do actually expect to see that happen.

Are search engines getting better or worse for you? It was a lot easier getting the right paper from a search engine 10 years ago. Now, you just get half a page of irrelevant ads, and another half page of links boosted by payola.

Just imagine what it will be like when there are literally MILLIONS of bad papers for each good paper. Then Billions. There is no finite limit to the amount of bullshit that LLMs can--and therefore will--output.

I strongly object to any proposal that would interrupt this essential process. If you (somehow) prevented me from distributing my preliminary results to other scientists, or from reading their preliminary results, then I would actively work around your proposal. Not because I hate you, but because science fundamentally requires access to these new ideas. Moreover, any researchers who waited for replication would fall behind and get scooped by the better-connected Ted-talk giving folks, who would obviously keep exchanging results.

However, it's obvious that when you say publication you don't mean it in the literal sense of communicating scientific results. Instead what you're trying to reform is academic credit. We all know that scientists [and the bureaucrats who fund them] love to find ways to measure research productivity, and they've chosen to use publications (in specific venues) as a proxy for that contribution.

And, following Goodhart's law, any measure that becomes a target ceases to be a good measure.

So the purpose of your proposal really has nothing to do with scientific publication for its first-order effects (i.e., distributing results to colleagues) but rather, you just want to reform the way that credit is allocated to scientists for purposes of funding, promotion, and general adulation (that's a joke.)

My suggestion is: if you want to reform the way credit is allocated, why not focus explicitly on that goal? There's no need to put in place complicated new systems that will make it harder for anyone to publish (and that real scientists will instantly work around.) Instead, just go to the NSF and NEH and various university administrations and convince them to stop hiring/funding/firing researchers using the metrics they currently use?

I think overall you'll find that this is still a tough sell, and a massive uphill battle. But it has the advantages of (1) addressing the actual problem you're concerned with, (2) not preventing researchers from doing their job, (3) has fewer overall points of failure, and (4) isn't subject to being worked around by funders/administrators who simply adopt new measures after you "reform" the publication system (e.g., switching to citation counts for arXiv works, or surveying other researchers for their opinion.)

re#2: Under my proposal, an researchers in an independent lab would have to read a paper to see how to design and conduct an experiment to replicate the results. And if it didn't reproduce, they don't get their paper published.

Given the stakes, don't you think researchers would exert every effort to make their paper as transparent and as easy-to-read, as possible? How carefully would they describe their experiment if they knew somebody was going to take their description and use it to check their work?

Re #3: Yeah, but again that's not a problem specific to my proposal. The same risk hangs over every employer-employee relationship.