Scientific Peer Review Is Broken – Fighting to Fix It with Anonymity • Hacker News Board

Re: Scientific Peer Review Is Broken – Fighting to Fix It with Anonymity

#21

Post by dougmccune » Wed, Dec 10, 2014, 9:27 PM UTC

The actual complaint filed shows pretty nasty behavior. Here's the full complaint: https://retractionwatch.files.wordpress.com/2014/10/filed-co...

Someone very clearly was trying to get Dr. Sarkar fired. The PubPeer comments are only one small aspect. But the person then also (I assume anonymously) emailed his new employer to allege fraud, and then went so far as to print out these allegations and stamped them with official looking nonsense about being from the an NIH investigation (which didn't exist), and distributed them throughout Sarkar's department mail boxes.

The complaint makes a decent case for why they think pretty much all the negativity directed at this researcher is likely from one angry person. They obviously can't prove that multiple anonymous comments are from the same person, but reading the content it certainly seems likely. What is assumed to be the same person then took things way past the line of what most of us would consider ethical.

There may very well be a place for anonymous calling out of potential research misconduct. But making fraud allegations anonymously online, then printing out those comments, trying to fake them to look like an official government inquiry, and physically delivering them to the researcher's boss at his place of work isn't the way to do that. This case might be one of those cases where the anonymity should be protected at all costs out of principle, but it's a really shitty case to wave your "we're the good guys" flag for.

Re: Scientific Peer Review Is Broken – Fighting to Fix It with Anonymity

#22

Post by ylem » Wed, Dec 10, 2014, 9:27 PM UTC

This is an interesting problem. On the one hand, there are merits to a referee being anonymous to the submitter. Part of this may be to avoid reprisal for younger referees, but even for more established referees, you may be freer to comment if you are anonymous to the author. BUT, you are not completely anonymous. Hopefully (though there have been some recent scandals related to this) an editor of the journal knows your work and has chosen to use you as a referee based on that. This can help to keep down some of the noise that another poster (lisper) mentioned. Also, the fact that the editor knows who you are may provide some constraint on how you may phrase a review as compared to if you were completely anonymous.

I don't think this is something one should issue a lawsuit over--but I also don't think that their proposal of completely anonymous review is at all useful.

Re: Scientific Peer Review Is Broken – Fighting to Fix It with Anonymity

#23

Post by [deleted] » Wed, Dec 10, 2014, 9:31 PM UTC

lisper wrote: post #17

dluan wrote: post #15

Earlier quoted context omitted.

Debatable on whether or not you can assume litigation is grounds for defamation. Even if it is true, it's not necessary that anonymity required the scientist to sue. I think the appeal for the case to drop is not to discourage discussion of anonymity in review, but that it isn't discussion. A jury panel that decides to outcome of a paper could make science worse off than through a panel of peers.

The suit it a red herring. The point is that anonymity makes it more likely that bogus criticism will be heard, just as non-anonymity makes it more likely that valid criticism will not be heard. Anonymity is not a quality filter.

[deleted]

Re: Scientific Peer Review Is Broken – Fighting to Fix It with Anonymity

#24

Post by kazinator » Wed, Dec 10, 2014, 9:40 PM UTC

> A prominent cancer scientist, unhappy with the attention his research papers have received on PubPeer, is suing some of our anonymous commenters for defamation

On the other hand, should anonymous commenters have the balance of power: in other words, say whatever they want with impunity, even if it actually is defamatory?

(Not saying that is the case in this situation, but in general).

The problem is that defamation is a legal concept, which can only be tried legally. So for instance, whereas a site can rigorously enforce rules which say that all comments are directed at the research material, and not at persons, and have a factual basis in that material, those measures cannot take away the right of someone, who feels they have been defamed, to take the matter to court (where they will almost certainly lose, which is neither here nor there).

You can't just create a site and declare it above the law, so to speak.

The only way to protect the identities of the anonymous is for the site to take responsibility for the statements it publishes: to assert that the anonymous statements are subject to rigorous standards of review, and when published, they in fact reflect the views of PubPeer, and PubPeer alone, and not of any anonymous persons (who do not publish any statements, but act only as sources of information).

Then if someone feels they have been the target of defamation, the defendant shall be PubPeer.

Re: Scientific Peer Review Is Broken – Fighting to Fix It with Anonymity

#25

Post by jessriedel » Wed, Dec 10, 2014, 9:41 PM UTC

Fede_V wrote: post #7

I think the increase in fraudulent papers getting published is a symptom, not the disease. The disease is that there are too many scientists doing research for a given research budget. Due to the ferocious competition for grant money, people are either sloppy and cut corners, or do whatever they need to do and outright cheat to publish in top tier journals. As a society, we need to make a decision about how much we w…

Do you agree that that's a "command and control" approach to the market for research labor? In other words, it sounds like you want to have the government set the number of researchers in the field rather than simply pay for the amount of research it wants, in the same way some governments (the USSR is the classical example) in the past have decided on the number of manufacturers in a certain industry, not just how much they want to pay for a certain widget.

All things being equal, such approaches are generally discouraged by economists. So presumably you think that the market for research labor is unusual, e.g. has some market failure. Could you describe what you think that is?

Basically, my angle is that, once you've answered this question, it's almost always better to try to fix the market failure directly rather than go full command and control. For instance, if you think that prospective PhD students aren't well informed about the long odds against getting a tenured position, then it would be better to educate those students rather than instituting a cap on the number of PhDs that are funded.

Re: Scientific Peer Review Is Broken – Fighting to Fix It with Anonymity

#26

Post by hibikir » Wed, Dec 10, 2014, 9:46 PM UTC

dalke wrote: post #16

vacri wrote: post #9

Fortunately, the First Amendment is on our side. It protects the right to anonymous speech. I don't understand how "The government cannot outlaw your speech" means "Your anonymity is protected by law from private parties". How does that interpretation come about?

http://en.wikipedia.org/wiki/Dendrite_International,_Inc._v.... . > "the court must balance the defendant's First Amendment right of anonymous free speech against the strength of the prima facie case presented and the necessity for the disclosure of the anonymous defendant's identity."

I don't think that opinion says what you think it says.

What is really being discussed there is if the government could compel Yahoo to unmask an anonymous user. It says nothing about actual anonymity guarantees among private parties. If Yahoo decided to hand over the account details, the court would not have intervened at all. So when you have an account with some service, any expectations on how the company will handle the data come from the TOS.

In a similar fashion, a company is free to delete any content you submit to their website. The government can't curtail your speech, barring a few exceptions, but a company certainly can, as long as the government is not involved.

Re: Scientific Peer Review Is Broken – Fighting to Fix It with Anonymity

#27

Post by jessriedel » Wed, Dec 10, 2014, 9:48 PM UTC

kazinator wrote: post #24

> A prominent cancer scientist, unhappy with the attention his research papers have received on PubPeer, is suing some of our anonymous commenters for defamation On the other hand, should anonymous commenters have the balance of power: in other words, say whatever they want with impunity, even if it actually is defamatory? (Not saying that is the case in this situation, but in general). The problem is that defamation…

This situation is reminiscent of websites hosting copyrighted material posted by anonymous users. Does the relevant law (DMCA, I think) say anything about defamation? A reasonable compromise might be that websites are held harmless so long as they respond to official defamation complaints in some prescribed manner. (Of course, this might not work as well because defamation is significantly less clear-cut than copyright violations, which already has plenty of cases of over-reach due to disputes over fair use, etc.)

Re: Scientific Peer Review Is Broken – Fighting to Fix It with Anonymity

#28

Post by jessriedel » Wed, Dec 10, 2014, 9:51 PM UTC

lisper wrote: post #4

Anonymity can't fix scientific peer review, it can only replace type I errors with type II errors. Instead of suppressing criticism that ought not to be suppressed, anonymity can (and often does) fail to suppress criticism that really ought to be suppressed because it is in fact false and defamatory. And indeed, what this article is really about is a lawsuit that alleges that this kind of error has in fact taken plac…

All systems are imperfect, but one imperfect system can be vastly better than another.

Re: Scientific Peer Review Is Broken – Fighting to Fix It with Anonymity

#29

Post by Florin_Andrei » Wed, Dec 10, 2014, 9:51 PM UTC

lisper wrote: post #4

Anonymity can't fix scientific peer review, it can only replace type I errors with type II errors. Instead of suppressing criticism that ought not to be suppressed, anonymity can (and often does) fail to suppress criticism that really ought to be suppressed because it is in fact false and defamatory. And indeed, what this article is really about is a lawsuit that alleges that this kind of error has in fact taken plac…

I fear anonymity will just raise the level of irrelevant noise a whole lot.

Re: Scientific Peer Review Is Broken – Fighting to Fix It with Anonymity

#30

Post by zmanian » Wed, Dec 10, 2014, 9:51 PM UTC

It seems plausible to imagine a system where PeerPub could retain their "published author to register

This model relies on t a blind cryptographic signature.(https://en.wikipedia.org/wiki/Blind_signature)

PeerPub generates a public/private key pair.

1. Alice wishes to register on PeerPub. She generates a Nonce N.

2. She blinds the nonce with a factor B.

3. She submits the blinded nonce and her identity to PeerPub. PeerPub checks her credentials and executes a blind signature of the number and returns it to Alice.

4.Alice now separately registers an account perhaps using a privacy protecting system like Tor. She uses the original N and the unblinded Signature.

5. PeerPub verifies N + signature and registers the account. PeerPub will have no way of linking N to the original credentials. PeerPub can record N and make sure it can only be used once to register an account.