Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Let’s make peer review scientific

nature.com

Post reply on HN
Print view View original

81–90 of 146 posts

Re: Let’s make peer review scientific

#81

Post by __mbm__ »

p4bl0 wrote: post #28
joelg wrote: post #25

Earlier quoted context omitted.

Everything is open-sourced, and we've tried to document as much as possible to make it easy for people to host their own instances! Alternatively, you can just create arbitrarily many drafts without publishing any of them. https://github.com/pubpub/pubpub

Running my own instance is a bit heavy just to try the platform :). But thanks for the tips, I can create draft and delete them without trashing your instance. The fact that it is open source is a major positive point. I suppose you have heard of SJS [1] which in a way is very close to what PubPub is, except I do not believe in SJS because of it being closed source: it is way too unsafe for a scientific community to…

> and there is a risk of Elsevier buying it to shut it down

Yup. I'm still stung by Mendeley's sale to Elsevier.

Re: Let’s make peer review scientific

#82

Post by kd0amg »

apathy wrote: post #79
yayday wrote: post #75

Earlier quoted context omitted.

> For example, giving authors / organizations a Bitcoin address where they can receive funds from individuals / organizations who want to support their research. That's a fantastic idea. Maybe we could call this "depository" of money to conduct research something like, hmmm, what's a good word… a grant ? > Also, awarding reputation to authors based on the level of peer review their research has successfully undergone…

> Maybe we could call this "depository" of money to conduct research something like, hmmm, what's a good word… a grant? Nah, that word is already in use for stagnant allocations of academic welfare to work on bullshit instead of transformative techniques (e.g. CAR T-cells, which NIH refused to fund for years). Need a new word to signify "money that is actually intended to produce results" instead of "a pension for ir…

> or NSF (whose yearly funding is dwarfed by the R&D funding at a single company)

If the worst thing you can say about the NSF is that they need more money, that makes it sound like GP has come up with a nice way to allocate the available funding towards particular research projects.

> It would be nice if you thought a little more carefully about what you wrote. The devil is in the details.

Details like how to get "crowdfunding" to put up enough money that "independent scientist" can be a full time job and not just a hobby for the odd few who somehow already have most of the needed lab facilities/equipment?

Re: Let’s make peer review scientific

#83

Post by Fomite »

mariusz79 wrote: post #16

I've got a simpler solution - scientific paper should not be considered valid until another team replicates the findings. That would quickly get rid of all of the fake results, plus would weed out all of the research that nobody really cares about. Also require that all data is shared, and that until your results are not verified or replicated by a team from another university/country you don't get any more funds.

What will you do for a study of a national birth cohort?

Re: Let’s make peer review scientific

#84

Post by apathy »

kriro wrote: post #46
suchsciencewow wrote: post #41

Earlier quoted context omitted.

How are you planning on addressing the fact that none of this matters in the slightest if academic career growth/incentives/reputation specifically revolves around publishing boring pdfs in established journals? What do I gain by sending my work to your platform where the first question anyone in my "target audience" will ask is "why didn't this get published in a real journal, whats wrong with it?", and I will get e…

I think it's a political problem. Most researchers and professors tend to be employed by the state. Lobby the state to give very high preference to open access publication. In an ideal world I'd read a sentence like this in a hiring guideline for state universities: "only research that is available to all citizens of this state shall be considered as a contribution by the candidate" Additionally for research project…

Christ this would be brilliant. The trouble is that you have otherwise-decent people like Kirsten Gillibrand completely in the pocket of RELX. You have massive COI and crap like the PACE trial who WILL NOT, under any circumstances, release their data, and who consider academic inquiries regarding said data (the foundation of any scientific publication) as "vexatious".

In short, you're better off trying to get policy changed at NIH and other, major funders of research to push openness (Google and major tech companies already are quite good about this), and only resorting to legal challenges when the satanically evil Elsevier and friends attempt to buy off legislators.

If you were wondering whether academic institutions typically stand up for scholars (as opposed to faucets of grant money and indirects, who can behave as they please), kindly refer to the following. http://boingboing.net/2015/12/11/what-will-it-take-to-get-mi...

Academia is not scholarship. The institutions of academia are in fact opposed to scholarship if it might make their senior "leaders" (often far removed from any scholarly pursuits they might once of had) look bad. Academia responds wonderfully to incentives, though, and that's where I do believe you have nailed it.

Decision: Accept with Minor Revisions

;-)

Re: Let’s make peer review scientific

#85

Post by Fomite »

yayday wrote: post #68
mariusz79 wrote: post #21

Earlier quoted context omitted.

it will, if you require that the data is shared and can be analyzed by another group. In some cases it will slow down the progress initially, but I think in the long term we would be getting much better results.

You're already moving your goalposts. Your first post said replicates the finding. If I simply post my results and someone else looks at the data and agrees with me then are they really replicating my finding or did they, oh what's the word… peer review my work and agreed?

I really dislike the idea that "Your code runs on your data" is a form of replication.

Re: Let’s make peer review scientific

#87

Post by jessriedel »

chmike wrote: post #56
jessriedel wrote: post #23

Earlier quoted context omitted.

Physicists have access to this if they combine the arXiv with Scirate. https://scirate.com/ Very few choose to, and very little discussion actually happens. There is something of a chicken-or-egg problem, but the site is sufficiently established in one community (quantum info) that if folks wanted to discuss papers they could. So I take it as weak evidence that they just don't have the incentives to do so. If a centr…

arxiv is not open. At least not as open as it should be in my opinion. But I fully agree on the chicken-or-egg problem. The main problem is bootsrapping such web service. I don't think that "anoiting" a forum/web service would be a good strategy. We need to keep open the possibility of a new and better system to take over. My feeling is that two fundamental requirements must be satisfied. 1. Anybody should be able to…

http://vixra.org/

Re: Let’s make peer review scientific

#88

Post by IndianAstronaut »

joelg wrote: post #20

Shameless plug: I'm working at the MIT Media Lab on the PubPub project ( http://www.pubpub.org ), a free platform for totally open publishing designed to solve a lot of these problems: One is peer review, which, as some have already mentioned, needs to be done in an open, ongoing, and interactive forum. Making peer review transparent to both parties (and the public) makes everyone more honest. Another is the incentiv…

JOVE, the journal of visualized experiments, is also a very interesting model.

Re: Let’s make peer review scientific

#89

Post by dflock »

jsprogrammer wrote: post #72
danieltillett wrote: post #30

Earlier quoted context omitted.

I am in Australia where the pool of potential grant reviewers is pretty small for most areas. When you have a grant success rate of 15% it is all too easy to nobble your competitors via the review process.

Why is science viewed as a competition there?

For exactly the same reasons it is everywhere in the world: because there isn't enough funding to go around - and the grant application process is explicitly designed to be competitive.

Re: Let’s make peer review scientific

#90

Post by arca_vorago »

I'll preface by saying I'm only a sysadmin with a love of science, but I learned the most about peer review and scientific publishing while working in biotech at a genetics company. At some point I realized I actually had to have some understanding of the science to properly admin it, and began reading lots of papers, and I slowly started to realize just how bad many papers are.

There are a few key issues:

1. Scientists who "collaborate" with other scientists but do a small fraction of actual work get their names on papers as number fodder. Anytime I hear "I've been published in over 1000 journal articles" now I generally become more skeptical.

2. Lack of reproducibility. Not only in the methods and the documentation of the methods, but also the fact that most things just simply aren't even tested by a third party.

3. Publishing. They have far too long locked up information the public deserves to know, which is bad enough, but then they do a bad job of it and allow bad science in. The largest part of the problem imho because they created the situation that semi forces the scientists into questionable paper writing tactics.

Post reply on HN