I don't think push some form of crowd intelligence would change much. I guess the best thing we could do is to create some conditions to let new nodes emerge and complete. I also believe better funding distribution mechanism/system will help a lot.
Scientific Peer Review Is Broken – Fighting to Fix It with Anonymity
51–60 of 73 posts
Re: Scientific Peer Review Is Broken – Fighting to Fix It with Anonymity
#52> A prominent cancer scientist, unhappy with the attention his research papers have received on PubPeer, is suing some of our anonymous commenters for defamation On the other hand, should anonymous commenters have the balance of power: in other words, say whatever they want with impunity, even if it actually is defamatory? (Not saying that is the case in this situation, but in general). The problem is that defamation…
This situation is reminiscent of websites hosting copyrighted material posted by anonymous users. Does the relevant law (DMCA, I think) say anything about defamation? A reasonable compromise might be that websites are held harmless so long as they respond to official defamation complaints in some prescribed manner. (Of course, this might not work as well because defamation is significantly less clear-cut than copyrig…
DMCA is for copyright infringement, not defamation. That's not to say it doesn't get abused in all sorts of "we want you to take X down" scenarios, however.
For safe harbor here, you'd be looking to the CDA, I believe. In that case, what matters is the level of editorial control you exercise over the site. The complaint alleges that they review every comment before posting, which if true, would seem to be a problem for them. My understanding of the CDA is that, somewhat perversely, you're better off having low standards than high ones with respect to user-submitted content.
That aside, the above is a layman's understanding of the relevant law, not legal advice. If you're ever in a situation where this information is important, please discuss it with a lawyer.
Re: Scientific Peer Review Is Broken – Fighting to Fix It with Anonymity
#53Earlier quoted context omitted.
Agree with a lot of what is being said here. Have seen vicious anonymous reviews in CS - as we all have - both due to someone misunderstanding the work but also due to people from "a different camp" simply disliking a given approach. This must be common in other fields as well. Someone said on this thread that anonymity is not a guarantee of quality - completely agree. The move towards "open reviewing" for conference…
I'm going to say something controversial. Minor reform of the referee system is treating the symptoms, not the disease. The disease is that there is no honest, mutual, voluntary exchange between a party that values a given research project and the party that produces it. Rather, we have federal bureaucrats handing out money taken from taxpayers willy-nilly via an old-boy network as described above. (The difference be…
I'm not sure what the one has to do with the other. My research is corporate-funded and my papers still get reviews from people who didn't read the paper carefully. Unless you're proposing to eliminate the concept of publishing research at all...
Re: Scientific Peer Review Is Broken – Fighting to Fix It with Anonymity
#54The actual complaint filed shows pretty nasty behavior. Here's the full complaint: https://retractionwatch.files.wordpress.com/2014/10/filed-co... Someone very clearly was trying to get Dr. Sarkar fired. The PubPeer comments are only one small aspect. But the person then also (I assume anonymously) emailed his new employer to allege fraud, and then went so far as to print out these allegations and stamped them with o…
I am also not sure (as was a commenter in a reply) what was objectionable about my first reply to you, but now that I see that I can no longer edit that reply anyway, I should link here to what PubPeer says in reply to the plaintiff's motion to force disclosure of the name of the PubPeer commenter(s).
http://retractionwatch.com/wp-content/uploads/2014/12/2014.1...
As an aside, I'll mention that the PubPeer lawyer is doing a much better job in filing pleadings, perhaps because PubPeer is a more above-board client, than Sarkar's lawyer is doing. Yeesh, what a mess the Sarkar complaint is. (Yes, I am a laywer, and I have read pleadings before and have a gut sense of which pleadings are convincing.) Note that the American Civil Liberties Union signed off as lawyers for PubPeer.
Re: Scientific Peer Review Is Broken – Fighting to Fix It with Anonymity
#55Dr. Mark Burgess just blogged "Why I stopped caring about peer review, and learned to love the work". http://markburgess.org/blog_peer.html
Re: Scientific Peer Review Is Broken – Fighting to Fix It with Anonymity
#56> A prominent cancer scientist, unhappy with the attention his research papers have received on PubPeer, is suing some of our anonymous commenters for defamation On the other hand, should anonymous commenters have the balance of power: in other words, say whatever they want with impunity, even if it actually is defamatory? (Not saying that is the case in this situation, but in general). The problem is that defamation…
As is always the case, the First Amendment does not give universal and blanket permission. There is are multiple tensions. The appeals court in Dendrite v. Doe guidelines for this matter are:
> (1) the plaintiff must make good faith efforts to notify the poster and give the poster a reasonable opportunity to respond; (2) the plaintiff must specifically identify the poster's allegedly actionable statements; (3) the complaint must set forth a prima facie cause of action; (4) the plaintiff must support each element of the claim with sufficient evidence; and (5) "the court must balance the defendant's First Amendment right of anonymous free speech against the strength of the prima facie case presented and the necessity for the disclosure of the anonymous defendant's identity."
In case of defamation, there's already a pretty high barrier for prima facie cause of action, and relatively broad protection for anonymous speech.
There's also little special about online communications. Anonymous speech, and the difficulties in balancing the different factors, have existed since before the founding of the country, when anonymous pamphlets like 'Common Sense' were used to press the call for independence.
Regarding "the site [must] take responsibility for the statements it publishes", this is wrong. Otherwise HN must be responsible for everything posted here, and we would have no HN. The relevant law is Section 230 of the Communications Decency Act of 1996, which grants pretty broad immunity to service providers. See http://en.wikipedia.org/wiki/Section_230_of_the_Communicatio... .
So: 1) it's well-established that anonymous commenters cannot say whatever with impunity, 2) defamation in this case is little different than the many previous cases on the topic, 3) no one seriously believes you can create a site and declare it above the law (assuming no jurisdiction problems), 4) existing case law is on the side of PubPeer, and 5) CDA gives PubPeer the ability to host anonymous comments without automatically being the target of a defamation suit.
Re: Scientific Peer Review Is Broken – Fighting to Fix It with Anonymity
#57The actual complaint filed shows pretty nasty behavior. Here's the full complaint: https://retractionwatch.files.wordpress.com/2014/10/filed-co... Someone very clearly was trying to get Dr. Sarkar fired. The PubPeer comments are only one small aspect. But the person then also (I assume anonymously) emailed his new employer to allege fraud, and then went so far as to print out these allegations and stamped them with o…
I am also not sure (as was a commenter in a reply) what was objectionable about my first reply to you, but now that I see that I can no longer edit that reply anyway, I should link here to what PubPeer says in reply to the plaintiff's motion to force disclosure of the name of the PubPeer commenter(s). http://retractionwatch.com/wp-content/uploads/2014/12/2014.1... As an aside, I'll mention that the PubPeer lawyer is…
The response from PubPeer's lawyer is certainly interesting, particularly the analysis of the images in question. But I wasn't trying to argue one way or the other about the actual merit of the accusations really. I was more pointing out that it seems fairly clear that someone, acting entirely anonymously, was set on getting this man fired. He did indeed lose a job (in fact two!), seemingly because of the efforts of this anonymous source, without any kind of formal inquiry or detailed analysis of misconduct. And the alleged actions of the anonymous attacker were entirely unprofessional and just plain shitty. Does the guy deserve to have his career destroyed? I have no idea, but trying to do so in such an underhanded mean-spirited way is just nasty. But this is academia after all...
Re: Scientific Peer Review Is Broken – Fighting to Fix It with Anonymity
#58The actual complaint filed shows pretty nasty behavior. Here's the full complaint: https://retractionwatch.files.wordpress.com/2014/10/filed-co... Someone very clearly was trying to get Dr. Sarkar fired. The PubPeer comments are only one small aspect. But the person then also (I assume anonymously) emailed his new employer to allege fraud, and then went so far as to print out these allegations and stamped them with o…
I am also not sure (as was a commenter in a reply) what was objectionable about my first reply to you, but now that I see that I can no longer edit that reply anyway, I should link here to what PubPeer says in reply to the plaintiff's motion to force disclosure of the name of the PubPeer commenter(s). http://retractionwatch.com/wp-content/uploads/2014/12/2014.1... As an aside, I'll mention that the PubPeer lawyer is…
I feel compelled to point out that it's not possible to downvote the people who reply to you. I'm certain that you don't intend to imply this, and I think most people here know this, but it does come up on occasion.
Re: Scientific Peer Review Is Broken – Fighting to Fix It with Anonymity
#59Earlier quoted context omitted.
I'm going to say something controversial. Minor reform of the referee system is treating the symptoms, not the disease. The disease is that there is no honest, mutual, voluntary exchange between a party that values a given research project and the party that produces it. Rather, we have federal bureaucrats handing out money taken from taxpayers willy-nilly via an old-boy network as described above. (The difference be…
I'm not sure what the one has to do with the other. My research is corporate-funded and my papers still get reviews from people who didn't read the paper carefully. Unless you're proposing to eliminate the concept of publishing research at all...
What one has to do with the other is that the entire system is broken, here's my explanation for why the system is fundamentally messed up, and my explanation for how it should be. Simply trying to change refeering by itself is treating the symptom, not the disease.
> Unless you're proposing to eliminate the concept of publishing research at all...
I think putting my research on my website would be just as good as publishing it in a conference proceeding. I'm not against sharing research and having it be public, but I think conference proceeidngs and journals in CS are of very little value. If people want my research, they can find it on my website. And I think this generalizes... if I want your research, I can (presumably) find it on your website.
Ultimately, it may be useful to have some kind of aggregation of what research is coming out in various subfields. Internet fora and the like would serve the purpose fine. But sure, having a group of referees and some way to screen research and highlight what is good could certainly be useful. All that stuff should develop organically as it is needed, though.
The model we have now, with peer review, journals made out of dead trees, etc. is a hold-over from a pre-Internet time when a few people in every sphere of life controlled the information that was disseminated, because it wasn't possible for things to be open, since we lacked the technology. That is no longer an issue. Of course, academics will hold onto their little racket as long as they can (probably indefinitely).
Re: Scientific Peer Review Is Broken – Fighting to Fix It with Anonymity
#60The actual complaint filed shows pretty nasty behavior. Here's the full complaint: https://retractionwatch.files.wordpress.com/2014/10/filed-co... Someone very clearly was trying to get Dr. Sarkar fired. The PubPeer comments are only one small aspect. But the person then also (I assume anonymously) emailed his new employer to allege fraud, and then went so far as to print out these allegations and stamped them with o…
You should read the last link in the article. The whole thing is a great read actually. But take a look especially at the results section that starts on page 22. It proves very convincingly that "for each of the 28 image-issues... evaluated,
strong evidence supports the conclusion that the images are not authentic or contain other
irregularities symptomatic of tampering": https://www.aclu.org/sites/default/files/assets/krueger_affi...