Let’s make peer review scientific • Hacker News Board

Re: Let’s make peer review scientific

#141

Post by apathy » Thu, Jul 07, 2016, 5:27 PM UTC

untilHellbanned wrote: post #139

apathy wrote: post #133

Earlier quoted context omitted.

No, much bigger. They're going to start requiring OA.

Peer review also needs a reboot. It's epically unhelpful and slows down publishing dramatically, Think they should mandate post publication review.

The review model in the WTC's new world order is open, as it currently is in the F1000 journals, and the Wellcome Trust pays the APC so there is no excuse for grantees not to publish. This slowly kicks away all the bullshit excuses that are presented for lack of scientific productivity so that the funder can objectively assess "what are these people doing with our money, and does that advance real, non-press-release science?" It's brilliant imho.

https://wellcome.ac.uk/news/why-were-launching-new-publishin...

Momentum is important. There's no reverse on a submarine. One of the biggest funders in the world has taken away a standard excuse for failing to openly report results, negative or otherwise. It is free to readers and free to authors funded by the Trust. And all of the reviewing process is out in the open.

This is a huge step. The only other funding body I can think of with similar weight is NIH, and (at least internally) they're starting to move in the same direction.

Re: Let’s make peer review scientific

#142

Post by apathy » Thu, Jul 07, 2016, 5:39 PM UTC

yayday wrote: post #123

apathy wrote: post #79

Earlier quoted context omitted.

> Maybe we could call this "depository" of money to conduct research something like, hmmm, what's a good word… a grant? Nah, that word is already in use for stagnant allocations of academic welfare to work on bullshit instead of transformative techniques (e.g. CAR T-cells, which NIH refused to fund for years). Need a new word to signify "money that is actually intended to produce results" instead of "a pension for ir…

I wasn't saying the current system is great, there's a lot wrong with it yes I agree. The impact factor thing is also a pretty silly metric to me as well, I agree with you there. The point I was trying to make, albeit sarcastically, was that the system the guy proposed is what we have today just without the extra hoops to jump through. Like I have absolutely no interest in maintaining a Bitcoin wallet or whatever nor…

Mostly agreed, although I did consider using an HPC allocation to mine bitcoins & hire work study students. But then it turned out that if you study interesting stuff and write the ad correctly, you'll have to beat them away with a stick. For good measure, I convinced one of our corporate patrons that they ought to pay for one of the students.

As far as extra hoops, it's not clear to me whether endless NIH paperwork bloat and ICMJE declarations are more or less onerous than crowdsourcing type stuff. I tend to think there must be a happy medium, but I could just be naive.

Re: Let’s make peer review scientific

#143

Post by untilHellbanned » Thu, Jul 07, 2016, 7:19 PM UTC

apathy wrote: post #141

untilHellbanned wrote: post #139

Earlier quoted context omitted.

Peer review also needs a reboot. It's epically unhelpful and slows down publishing dramatically, Think they should mandate post publication review.

The review model in the WTC's new world order is open, as it currently is in the F1000 journals, and the Wellcome Trust pays the APC so there is no excuse for grantees not to publish. This slowly kicks away all the bullshit excuses that are presented for lack of scientific productivity so that the funder can objectively assess "what are these people doing with our money, and does that advance real, non-press-release…

I'm a new professor in the US so I'm happy for you but it looks like we'll have to wait for the NIH to do its job.

F1000 is entirely post-publication review where reviewers identities are known? If so, I love it!

Re: Let’s make peer review scientific

#144

Post by alpineidyll3 » Thu, Jul 07, 2016, 10:53 PM UTC

joelg wrote: post #20

Shameless plug: I'm working at the MIT Media Lab on the PubPub project ( http://www.pubpub.org ), a free platform for totally open publishing designed to solve a lot of these problems: One is peer review, which, as some have already mentioned, needs to be done in an open, ongoing, and interactive forum. Making peer review transparent to both parties (and the public) makes everyone more honest. Another is the incentiv…

No need for shame with such a worthy plug+!!!!

Re: Let’s make peer review scientific

#145

Post by apathy » Fri, Jul 08, 2016, 4:08 AM UTC

untilHellbanned wrote: post #143

apathy wrote: post #141

Earlier quoted context omitted.

The review model in the WTC's new world order is open, as it currently is in the F1000 journals, and the Wellcome Trust pays the APC so there is no excuse for grantees not to publish. This slowly kicks away all the bullshit excuses that are presented for lack of scientific productivity so that the funder can objectively assess "what are these people doing with our money, and does that advance real, non-press-release…

I'm a new professor in the US so I'm happy for you but it looks like we'll have to wait for the NIH to do its job. F1000 is entirely post-publication review where reviewers identities are known? If so, I love it!

The NIH does a pretty decent job all things considered. With an organization that size, there is considerable inertia, but newer blood is starting to fix some of the older problems.

Best of luck -- work on stuff that matters, if you can :-)

Re: Let’s make peer review scientific

#146

Post by alphydan » Mon, Jul 11, 2016, 2:32 PM UTC

Fomite wrote: post #83

mariusz79 wrote: post #16

I've got a simpler solution - scientific paper should not be considered valid until another team replicates the findings. That would quickly get rid of all of the fake results, plus would weed out all of the research that nobody really cares about. Also require that all data is shared, and that until your results are not verified or replicated by a team from another university/country you don't get any more funds.

What will you do for a study of a national birth cohort?

Clone them?