The current academic peer review system is an example of a process that worked fine when number of participants was small but fails at scale. I'd like to see something like hacker news or reddit's social version of vetting papers in addition to any formal review process, perhaps with additional moderation from known experts. There is currently no site for community review of articles or papers and I think it would be a great resource for scientists of all kinds.
Publishing: The peer-review scam
11–20 of 45 posts
Re: Publishing: The peer-review scam
#12The fundamental issue with the current peer review system is that most academic research is "valueless" to those involved in the system. I use the term "valueless" in a very careful and specific meaning. It does not mean the work is without merit or doesn't have large impact or benefits to society, rather it simply means the quality of the work has no value to the peer reviewers or to the journal itself. * The peer r…
I thought one of the main points of academia was that it isn't centered around making money.
Re: Publishing: The peer-review scam
#13The fundamental issue with the current peer review system is that most academic research is "valueless" to those involved in the system. I use the term "valueless" in a very careful and specific meaning. It does not mean the work is without merit or doesn't have large impact or benefits to society, rather it simply means the quality of the work has no value to the peer reviewers or to the journal itself. * The peer r…
There is, or should be, more to academia than just the pursuit of money.
Re: Publishing: The peer-review scam
#14The fundamental issue with the current peer review system is that most academic research is "valueless" to those involved in the system. I use the term "valueless" in a very careful and specific meaning. It does not mean the work is without merit or doesn't have large impact or benefits to society, rather it simply means the quality of the work has no value to the peer reviewers or to the journal itself. * The peer r…
No. Peer reviewers get: early access to articles of interest, the feeling of contributing to the community they belong to, the ability to demonstrate their knowledge, the ability to reciprocate other people for reviewing their own articles, and so on. These are not financial rewards, but they are not less strong for that (arguably even stronger).
Journals are heavily incentivized to maintain standards because otherwise good authors will stop submitting to them and librarians will stop subscribing (of course only the first incentive holds for OA journals).
The fact that system sometimes fails is not proof that it is a bad system - all human systems have a failure rate.
Re: Publishing: The peer-review scam
#15This is one of the problems we're trying to solve with Publons.com. We help reviewers to build an officially verified record of past reviews (which is great when it comes to getting tenure). Editors can then use those records to vett candidate reviewers.
Re: Publishing: The peer-review scam
#16[reposting from a previous submission that didn't get any traction] If anyone wants a good idea for a startup in the academic journals space, I'd recommend looking into vetting peer reviewers as part of the article submission/review process. There have been a few high profile cases of people faking their peer reviews recently. At Sage we dealt with a big case of that earlier in the year and retracted 60 papers. A new…
The OA EGU Journals have been doing a good job of reviewing. The review process is open to the public with anyone able to read and comment on a paper, while they also invite/require several experts to participate. While some papers are only reviewed by the invited reviewers, many are reviewed by 5+ people. Having the entire process open to the public is the way forward.
Re: Publishing: The peer-review scam
#17> researchers exploited vulnerabilities in the publishers' computerized systems
...
> But even the most secure software could be compromised.
This is nonsense, it is nothing to do with software vulnerabilities. It would be no more or less "secure" if I mailed in printed paper with a cover letter saying "Thomas Edison would be a suitable reviewer, his phone number is 555 5555 5555".
Re: Publishing: The peer-review scam
#18Disclaimer: I haven't read the whole article, so maybe it was mentioned there. My idea: only allowing reviews from (primary) authors of already-published, peer-reviewed journals? Bootstrapping wouldn't be a problem, because there are already quite many papers published, many of them containing the email address of the authors. It would make it much harder to set up fake identities for peer reviews.
This is the existing system. It would be extremely rare for someone to be asked to review without having already published.
Re: Publishing: The peer-review scam
#19The entire scam seems to be possible because there is no attempt at confirming the credibility of the people presuming to actually do the peer review. This type of decentralized authentication is precisely what PGP is particularly good at. I have to imagine that it would be trivial to design a system which would simultaneously ensure that only the actual scientists would be able to peer review a paper based on other…
If you image this is a trivial problem to solve, I think your imagination is not very good. I'll quote the Bruce Schneier quote from the article. “There are almost never technical solutions to social problems.”
I don't know if the problem is trivial to solve, but designing a system which engages one element of the problem is trivial. The part that I was focusing on, was ensuring that any paper that claims to be Reviewed by a scientist actually was reviewed by that scientist.
Re: Publishing: The peer-review scam
#20The fundamental issue with the current peer review system is that most academic research is "valueless" to those involved in the system. I use the term "valueless" in a very careful and specific meaning. It does not mean the work is without merit or doesn't have large impact or benefits to society, rather it simply means the quality of the work has no value to the peer reviewers or to the journal itself. * The peer r…
There is, or should be, more to academia than just the pursuit of money.
Of course. The primary goal of academia is the advancement of human knowledge.
Unfortunately, the current ecosystem of journals is to a significant extent failing at that. Possibly introducing a clear [0] profit motive into that system would result in it working more efficiently and better serving academia's primary goal.
[0] I say "clear" because a lot of the current system of journals is primarily driven by profit motives that are only slightly veiled. There are notable exceptions to this, e.g. PLoS, Journal of Statistical Software, etc...