Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Let’s make peer review scientific

nature.com

Post reply on HN
Print view View original

131–140 of 146 posts

Re: Let’s make peer review scientific

#131

Post by apathy »

jsprogrammer wrote: post #121
apathy wrote: post #109

Earlier quoted context omitted.

Every time you use the word "should" you beg the question. Why, exactly, are incentives aligned to make it so? Or is the problem that the incentives are not in fact set up to encourage the desired outcome? Anything else is wishful thinking. People seldom do what they know to be right; they do what is convenient, then repent. It follows that one must make the right thing the convenient or rewarding thing if one wishes…

Reviews should be facts by definition of a review (as I use it), so the question is not begged. The incentives are 'aligned' that way because it is the only way to benefit. Publishing into an echo chamber (or void) is not science, by definition.

You've been out of academia for a while, I take it?

Publons (mostly uncited) are the coin of the realm. I don't usually bother submitting a methods paper unless I know a few citations are in the tank, but some people publish just to get their metrics up. A lot of people, really. The more I think about it the luckier I feel not to have to continually publish jackoff papers. If I wasn't so lucky in terms of collaborators... Well, I guess I'd GTFO of academia, really.

Lots of folks have to "show productivity" with bullshit papers that don't matter and no one reads. Needless to say, reviewing such papers can be a rather dull affair.

Re: Let’s make peer review scientific

#132

Post by [deleted] »

jsprogrammer wrote: post #121
apathy wrote: post #109

Earlier quoted context omitted.

Every time you use the word "should" you beg the question. Why, exactly, are incentives aligned to make it so? Or is the problem that the incentives are not in fact set up to encourage the desired outcome? Anything else is wishful thinking. People seldom do what they know to be right; they do what is convenient, then repent. It follows that one must make the right thing the convenient or rewarding thing if one wishes…

Reviews should be facts by definition of a review (as I use it), so the question is not begged. The incentives are 'aligned' that way because it is the only way to benefit. Publishing into an echo chamber (or void) is not science, by definition.

[deleted]

Re: Let’s make peer review scientific

#133

Post by apathy »

untilHellbanned wrote: post #127
apathy wrote: post #105

Earlier quoted context omitted.

They have one (several, actually). The trouble is that they're called Cell, Nature, and Science; they're all for-profit; and they answer to no one. The Wellcome Trust is walking the walk that NIH talked for the past decade. We will see how this goes. Cross fingers.

With ELife?

No, much bigger. They're going to start requiring OA.

Re: Let’s make peer review scientific

#134

Post by apathy »

quadrangle wrote: post #128
apathy wrote: post #99

Earlier quoted context omitted.

not really https://github.com/overleaf Some parts of it are (duh?) but overall the system is astoundingly transparent. I think they ended up charging for the sync-to-git option, but if you look at the above repos, it appears that this feature can actually be pulled on the user end rather than just pushed from Overleaf. Overleaf integrates nicely with existing journals and with preprint servers to some extent; I've bu…

not sure how to parse all that. At the least, it's not straightforward, and my original comment is 100% true: "that appears to be totally proprietary" (emphasis added) If a project is actually FLO, you'd think they'd make that clear to someone visiting their website.

They're a corporation. Some of their decisions are made accordingly. The most interesting bits, though (version controlled dumps of the LaTeX you're working on) do have open implementations.

Ultimately, it (like Paperpile) is a service that is built from mostly open components, integrates nicely with OA workflows, and has no real lock-in. I'm not sure what you're afraid of -- a paper will unsubmit itself? LaTeX will go closed source? Since your entire version history is there for the taking at any time, there's not much downside, and for our manuscripts & tech reports we've seen huge upside.

Ymmv!

Re: Let’s make peer review scientific

#135

Post by apathy »

jsprogrammer wrote: post #130
dflock wrote: post #89

Earlier quoted context omitted.

For exactly the same reasons it is everywhere in the world: because there isn't enough funding to go around - and the grant application process is explicitly designed to be competitive.

It doesn't seem to be everywhere though. I know in the US, research is often collaborative and a single grant may fund many researchers. Those researchers do not seem to be competing internally for funding (though, I have seen conflict as to whose name should appear first on a publication!). I presume the same is likely true in Australia. Why is some research able to be collaborative while other must (apparently) be…

You are quite mistaken, imho. The pay line for NIH grants is around 15% and for NSF grants it's more like 10%.

Re: Let’s make peer review scientific

#136

Post by apathy »

jsprogrammer wrote: post #130
dflock wrote: post #89

Earlier quoted context omitted.

For exactly the same reasons it is everywhere in the world: because there isn't enough funding to go around - and the grant application process is explicitly designed to be competitive.

It doesn't seem to be everywhere though. I know in the US, research is often collaborative and a single grant may fund many researchers. Those researchers do not seem to be competing internally for funding (though, I have seen conflict as to whose name should appear first on a publication!). I presume the same is likely true in Australia. Why is some research able to be collaborative while other must (apparently) be…

Note also that collaborations compete with other collaborations for program grants, and R01s (the measure of a "real" PI) are inherently not collaborative. They are by their very nature competitive. And they are the metric by which principal investigators are judged.

Re: Let’s make peer review scientific

#137

Post by jsprogrammer »

apathy wrote: post #136
jsprogrammer wrote: post #130

Earlier quoted context omitted.

It doesn't seem to be everywhere though. I know in the US, research is often collaborative and a single grant may fund many researchers. Those researchers do not seem to be competing internally for funding (though, I have seen conflict as to whose name should appear first on a publication!). I presume the same is likely true in Australia. Why is some research able to be collaborative while other must (apparently) be…

Note also that collaborations compete with other collaborations for program grants, and R01s ( the measure of a "real" PI) are inherently not collaborative. They are by their very nature competitive. And they are the metric by which principal investigators are judged.

Many real PIs support collaborating researchers and students though.

Re: Let’s make peer review scientific

#138

Post by wodenokoto »

joelg wrote: post #20

Shameless plug: I'm working at the MIT Media Lab on the PubPub project ( http://www.pubpub.org ), a free platform for totally open publishing designed to solve a lot of these problems: One is peer review, which, as some have already mentioned, needs to be done in an open, ongoing, and interactive forum. Making peer review transparent to both parties (and the public) makes everyone more honest. Another is the incentiv…

I skimmed through the site and am left with the question: where do I submit my paper?

Re: Let’s make peer review scientific

#139

Post by untilHellbanned »

apathy wrote: post #133
untilHellbanned wrote: post #127

Earlier quoted context omitted.

With ELife?

No, much bigger. They're going to start requiring OA.

Peer review also needs a reboot. It's epically unhelpful and slows down publishing dramatically, Think they should mandate post publication review.

Re: Let’s make peer review scientific

#140

Post by apathy »

jsprogrammer wrote: post #137
apathy wrote: post #136

Earlier quoted context omitted.

Note also that collaborations compete with other collaborations for program grants, and R01s ( the measure of a "real" PI) are inherently not collaborative. They are by their very nature competitive. And they are the metric by which principal investigators are judged.

Many real PIs support collaborating researchers and students though.

Right, but students are written into the grant. Ultimately, nobody gives a shit which post doc or student you put on a project; the assumption (often flawed) is that they're all the same as far as the modular budget is concerned.

If they were special, the reasoning goes, they'd have their own F32 or T32 to work on the project.

Post reply on HN