Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The case against peer review

slate.com

Post reply on HN
Print view View original

11–17 of 17 posts

Re: The case against peer review

#11

Post by vacri »

dspeyer wrote: post #6

If not peer review then what? Centralized review? Is there any hope that that won't collapse under its own weight? Public review? That hasn't done all that well in the Open Source world.

Come now. Out of literally thousands of peer-reviewed articles that hit the journals every month, the author provides one example of a bad paper from 2001 and some vague handwaving at problems in the early '80s.

Clearly this means that the system is a total and utter failure and should be tossed ASAP.

Leaving the sarcasm behind now, peer review is but one element in the scientific process - and okay, it failed for the example paper. But another element - free and frank community feedback (ie: no sacred cows) - killed the paper, by the author's own admission. It's but one element of a series of checks and balances.

Besides, although I've only published one paper, the feedback from the peer reviewers made that paper tighter and more professional - so for my solitary data point, that element of the process did improve the quality of the article.

Re: The case against peer review

#14

Post by d0mine »

ohmygodel wrote: post #4
bmastenbrook wrote: post #2

My experience being on the reviewer side of the process is very limited. As a frequent reader of papers in CS I'm of the opinion that peer review is accomplishing very little outside of checking theoretical results with proofs that can be vetted. I can count on one hand the number of papers I've read in the past month or two describing an implementation of a technique where the implementation described was available…

"As a frequent reader of papers in CS I'm of the opinion that peer review is accomplishing very little outside of checking theoretical results with proofs that can be vetted." As a frequent reader and reviewer of CS papers I totally disagree. Peer review helps narrow a huge incoming flood of new research into a smaller stream of results that are the most novel, important, and correct. Without explicit review, your op…

The internet by itself can play role of the peer review system. For example, github is a much better medium to publish a work that needs to be examined (reproduced) than any paper journal.

Re: The case against peer review

#16

Post by lotharbot »

roadnottaken wrote: post #5

Peer review is not "the gold standard of modern science." It is the very first step in a long process of communal collection and evaluation of data. Everyone knows it's badly flawed, but it's better than nothing. It's only when scientific results are reproduced and (more importantly) built-upon that an experiment or finding becomes widely accepted (and thus "true"). The problem comes in when the media pounces on ever…

Right. Peer Review doesn't mean "this paper is correct", it merely means "this is interesting enough to be worth a look." It's a way of filtering out the clearly bogus and the trivial/repetitive. True progress starts after peer review and publication, when other scientists look at the same model, write their critiques, and do their own experiments.

I'd think this would be old hat to anyone who went to grad school in a technical field. I know in my department, all students were expected to read and discuss several peer-reviewed papers each term. Finding methodological weaknesses, oversimplifications, or outright mistakes was common.

Re: The case against peer review

#17

Post by Locke1689 »

bmastenbrook wrote: post #2

My experience being on the reviewer side of the process is very limited. As a frequent reader of papers in CS I'm of the opinion that peer review is accomplishing very little outside of checking theoretical results with proofs that can be vetted. I can count on one hand the number of papers I've read in the past month or two describing an implementation of a technique where the implementation described was available…

In my experience, the biggest problem in CS academia is reproduction. It seems that very few people spend time attempting to reproduce results from academic papers, even though that's an important part of the method.
Post reply on HN