Peer review: the end of an error? • Hacker News Board

Re: Peer review: the end of an error?

#31

Post by jessaustin » Wed, Mar 21, 2018, 9:06 PM UTC

xamuel wrote: post #29

Not to defend peer review as it is now, but one of the things I've come to value from editorial gatekeepers is that they put the brakes on me publishing things that in later years I might regret. I wrote some pretty mediocre papers as a grad student and I'm grateful some of them were rejected, because I wouldn't want them forever part of my legacy. As a matter of fact, I wrote some "papers" when I was a teenager , wh…

Do some of the alternative platforms offer a way to repudiate a "published" paper?

Re: Peer review: the end of an error?

#32

Post by danieltillett » Wed, Mar 21, 2018, 9:09 PM UTC

chrisseaton wrote: post #4

This article says that peer review is unpaid - it isn't. If you are an academic or work in industrial research, it's part of your paid job to do peer review as part of the community. You are paid for it - it's part of your job and it's normal to do it during work hours.

Trust me that a minute spent reviewing a paper is one less minute you have toward your actual KPI’s. About the only value of peer review is to give the paper to your students and tell them to find all the mistakes and holes in the paper. This is actually quite a good for the education of grad students as they learn what not to do.

Re: Peer review: the end of an error?

#33

Post by xamuel » Wed, Mar 21, 2018, 9:09 PM UTC

danieltillett wrote: post #30

xamuel wrote: post #29

Not to defend peer review as it is now, but one of the things I've come to value from editorial gatekeepers is that they put the brakes on me publishing things that in later years I might regret. I wrote some pretty mediocre papers as a grad student and I'm grateful some of them were rejected, because I wouldn't want them forever part of my legacy. As a matter of fact, I wrote some "papers" when I was a teenager , wh…

Didn’t your supervisor step in and say this is not up to scratch? When I was a supervisor a good chunk of my time was spent going through my student’s and postdoc’s manuscripts to ensure nothing was in them that might embarrass us both.

I was reckless and wrote papers unrelated to my dissertation and submitted without going through my advisor.

Fortunately, the bad ones were rejected. The good ones ended up making me well-recognized in some extremely tiny niches, and ultimately did way more for my career than my PhD work did :D I'm really glad some of the bad ones didn't see the light of day :P

Re: Peer review: the end of an error?

#34

Post by kolpa » Wed, Mar 21, 2018, 9:10 PM UTC

The academic peer review debate has strong parallels to the social-media news debate -- As traditional newspapers/ TV channels are no longer gatekeepers of publication, we have given up a level of vetting in exchange for freedom-from-censorship. But that puts an onus on consumers -- it's not right for consumer to accept everything they see with the same level of default trust they had before. Consumers still must "consider the source", and society to create an effective replacement for the editorial filters we are discarding.

Re: Peer review: the end of an error?

#35

Post by kolpa » Wed, Mar 21, 2018, 9:12 PM UTC

heckless wrote: post #12

chrisseaton wrote: post #10

Earlier quoted context omitted.

This is such an adversarial way to look at it. I see it as a community. We write and share our papers as part of our jobs, we review each others' papers as part of our jobs, we help to run the conference as part of our jobs. Everyone contributes to run the system. Nobody buys articles for $35 a piece. All major universities and companies subscribe to the ACM or whatever your field's organisation is for you. And guess…

I don't have a huge issue with the ACM or even so much with IEEE. But have you ever submitted to or reviewed for Springer or Elsevier? It's an emasculating and thankless experience, and Springer and Elsevier go get to make huge profits off of everyone's free work. (haha, we just did it for exposure, right???) I don't mean to imply that peer review is this shitty thing that we "have" to do. I review papers regularly a…

> emasculating

that's an unfortunate word.

Re: Peer review: the end of an error?

#36

Post by phalangion » Wed, Mar 21, 2018, 9:14 PM UTC

woliveirajr wrote: post #3

> Remarkably, we have arrived at a system where academics feel a moral obligation to perform the thankless task of reviewing the work of other academics, anonymously and unpaid. This undoubtedly makes the literature better than it would otherwise have been, and ensures at least one reader for each paper. I'm afraid it doesn't ensure that all papers will be read. Some papers can attract more attention, some papers can…

I think the "one reader" refers to the reviewer, who ostensibly should read as a part of the review process.

Re: Peer review: the end of an error?

#37

Post by danieltillett » Wed, Mar 21, 2018, 9:20 PM UTC

The big problem with peer review is almost no reviewing takes place and it is at best is either a way for your enemies to get revenge or a cheap proofreading service.

The solution is to offer a largish reward to the most constructive peer review for each manuscript chosen by the editor, or better by other reviewers. If I (or more likely one of my starving grad students) knew that I would get $1000 for providing the most constructive peer review then I would put in the required effort to make peer review a worthwhile exercise.

One thing that would help the peer review process is a data dump of the raw data. Many times I have seen some figure or table in a manuscript and felt I could not provide a constructive critic without seeing the underlying data. Going back through the editor to get this data is just too much of a hassle most of the time.

Re: Peer review: the end of an error?

#38

Post by danieltillett » Wed, Mar 21, 2018, 9:27 PM UTC

xamuel wrote: post #33

danieltillett wrote: post #30

Earlier quoted context omitted.

Didn’t your supervisor step in and say this is not up to scratch? When I was a supervisor a good chunk of my time was spent going through my student’s and postdoc’s manuscripts to ensure nothing was in them that might embarrass us both.

I was reckless and wrote papers unrelated to my dissertation and submitted without going through my advisor. Fortunately, the bad ones were rejected. The good ones ended up making me well-recognized in some extremely tiny niches, and ultimately did way more for my career than my PhD work did :D I'm really glad some of the bad ones didn't see the light of day :P

mmm not wise :)

I guess it depends on the field, but in mine (biological sciences) I can’t think of an example where a student would submit a paper without their supervisor knowing.

Anyone thinking of doing the same as you should try and draw on their colleagues in the department to proofread the manuscript. I used to proof lots of manuscripts from my colleagues and their students.

Re: Peer review: the end of an error?

#39

Post by arca_vorago » Wed, Mar 21, 2018, 9:32 PM UTC

I've said it before and I'll say it again.

The focus should be on reproducing studies, not on peer-review.

Of course peer review can be of assistance in selecting studies worth putting the effort and funding into reproducing, but peer-review itself is extremely corrupted as a process, but since so many peoples careers depend on it , many of them will not talk about those issues.

I used to hold the peer-review and journal process on a pedestal. Then when working at a genetics lab I ended up having to read a lot of papers to get my sysadmin work done, and started realizing just how bad so many papers are, even though published in "reputable" journals.

For example, since publishing is seen as so vital to some people, I noticed many of them would "consult" on another groups paper just to get their name on the paper, even though they did almost no work, and then those same people would turn around and say stuff like "I've been published X times!" as a sort of reinforcement of authoritativeness (and as advertisement).

People are gaming the system because hardly anyone is actually checking them. Beyond that, the actual peer-review process itself seems to be very lackluster in rigor on it own.

I think there should be a certain amount of public research funding that should be earmarked to go directly to independent reproducability labs.

On top of it all, all this research that the public is funding is often behind a fucking paywall... and that's not just bullshit, it's theft and misuse of public funds. It should be required that publicly funded research should be publicly published, and therefore available for more eyes to see and review.

Re: Peer review: the end of an error?

#40

Post by dragonwriter » Wed, Mar 21, 2018, 9:39 PM UTC

arca_vorago wrote: post #39

I've said it before and I'll say it again. The focus should be on reproducing studies, not on peer-review. Of course peer review can be of assistance in selecting studies worth putting the effort and funding into reproducing, but peer-review itself is extremely corrupted as a process, but since so many peoples careers depend on it , many of them will not talk about those issues. I used to hold the peer-review and jou…

> The focus should be on reproducing studies, not on peer-review

The two aren't separable, since a key part of peer review is assuring that a published study is, in principle, reproducible. You can't focus on reproduction withoit focussing on peer review.