Replace peer review with “peer replication” (2021)

as far as I can tell, you weren't asked to prove either climate change or a round earth, or any concept at all

recall the tangent we deflected to was the mere existence of round earth deniers, climate change deniers (read: people), both of which do, in fact, exist as real people in the real world, as can be easily confirmed by anyone researching the topic in good faith, like I said

anyhow, back on topic: the proposed alternative just doesn't seem better than the status quo, no matter how you've sliced it so far, for the reasons given in my original post and ignored by you

you weren't able to do so, since "X is ridiculous to deny" is orthogonal to "there exist people who deny X", and the latter is the tangent deflected to. Like I said, anyone actually spending ten seconds researching the topic in good faith would discover that both round-earth denialists and climate-change denialists exist in real life, no matter how ridiculous their beliefs

anyhow, back on topic: the proposed alternative just doesn't seem better than the status quo, no matter how you've sliced it so far, for the reasons given in my original post and ignored by you

Well there isn't a way to fix this replicative difficulty for many fields. So you're suggesting that the theoretical understanding gained from having produced a specific material just never be added to scientific understanding at all, ever. In an ideal world with ideal budget allotment there would be time and money to build entire replicating labs that can be moved and modularly reconfigured with absolute control over air particulates, pressure, flow, sufficient to replicate any room possible anywhere on earth. But if a lab in the himalayas was able to make a magnet that fundamentally reconfigures our understanfing of physical phenomena, we should simply abandon that potential deeper truth into were able to build a second lab in the Himalayas, or in the middle of the Atlantic.

Reproduction is hard, really really fucking hard. Just saying, that means we should replicate before trying to understand, means essentially cutting off the understanding.

And like others have said, if someone wants to build from it they'll depend on that information being correct, if no one can manage to ever build from it then the idea dies.

Also there's a huge difference between, replicate this study on infant response to stimulus, or spider colony behavior, and, replicate this incredibly intricate semiconductor that took years of configuration to correctly produce.

> That's not a true replication

As you already quoted me as saying, it's not a perfect replication. Which is fine! I'm advocating a position of "replicate findings as much as reasonably possible", not a position of "we need to build redundant copies of every multi-billion-dollar research megaproject". My whole point is that this doesn't need to be an absolutist true-or-false sort of thing.

> and it isn't going to avoid issues like the OPERA experiment measuring neutrinos going faster than the speed of light due to a loose connector

Maybe not. I never claimed this would solve every problem in all of science forever.

> So all currently-running long-running research has to be thrown out?

No. I think it's reasonable to propose more rigorous standards for future studies without throwing out every in-progress study that didn't follow those same standards. After all, there are literally centuries of published science that didn't even follow the contemporary standards of peer review, and we haven't thrown any of that out.

> What if the two studies find very small differences, are you allowed to publish either of them?

That's an extremely broad question. You might as well ask, "what does it mean for a finding to be replicated?".

If you and I each independently go out to measure the length of the Golden Gate Bridge in millimeters, there are likely to be very small differences in the result you get and the result I get. There's an expected margin of error here where we can agree that our results are consistent with each other. Sometimes the differences are reasonable and can be explained, and sometimes they can't be explained.

Regardless, I also think it might even be valuable for the studies to be published in some form even if they don't replicate at all; just not necessarily with the imprimatur of some credible or prestigious journal.

> Are the two teams allowed to collaborate at all?

I would suggest keeping the two teams independent at least until they both have results. Afterwards, it might be valuable for the team to collaborate in terms of trying to reconcile their results.

> You can't "replicate" an observation of a freak astronomical event because you can't trigger a freak astronomical event. At best you can do observations and hope it happens again. We indeed cannot draw any conclusions from it, but that doesn't mean you can't publish papers about it. If replication is mandatory, you would not be allowed to do anything with it at all.

Yeah, I guess I'm fine publishing a paper that just says "here's this anomalous observation we had" if it's an especially interesting anomalous observation like that.

> It cannot be reproduced because it would be unethical to expose people to near-fatal levels of radiation simply for reproduction.

Obviously. I think you know that's not what I'm suggesting at all here.

> Simply reusing data from the original test subjects isn't a reproduction, after all.

At this point, I agree we're mostly stuck with whatever data we managed to get 37 years ago. But if a similar incident happened in the future, you could have independent teams collecting redundant sets of data.