Live data from Hacker News

Scientific integrity and the ethics of 'utter honesty'

thereader.mitpress.mit.edu

21–30 of 35 posts

Re: Scientific integrity and the ethics of 'utter honesty'

#21
post #5

There actually was a Journal of Negative Results in Biomedicine, but it ceased publication in 2017.

Since 2019, there exists "The Conference for Failed Approaches and Insightful Losses in Cryptology", a.k.a. CFAIL [0]

[0]: https://www.cfail.org/

Re: Scientific integrity and the ethics of 'utter honesty'

#22

I think the most important quotes from Feynman's address are the ones about being fooled, such as this one: > The first principle is that you must not fool yourself—and you are the easiest person to fool. So you have to be very careful about that. After you’ve not fooled yourself, it’s easy not to fool other scientists. You just have to be honest in a conventional way after that. [0] [0] https://calteches.library.cal…

"> The first principle is that you must not fool yourself—"

Good post. From my experience, it's pretty easy to fool oneself (it's happened to me too many times).

It's very easy for one to have preconceived notions or ideas about something and as such it doesn't take much evidence if any for one to be convinced that one's ideas about them are correct even if factually they're to the contrary.

Moreover, precovinced ideas and notions often stop one from reviewing existing evidence about them that one already has in one's possession. When one's without supporting evidence then the very nature of that preconception may stop one from seeking out new or additional information about them.

Essentially, one's entrenched views that one hasn't come by through a process of substantial and rigorous thought one ought to always question.

The trouble is it's easy to say that, to keep the notion ever present at the forefront of one's mind is damn hard.

Re: Scientific integrity and the ethics of 'utter honesty'

#23
post #18

I think this article kind of misses the mark? There is a big gulf between reporting other explanations and tests for the phenomena you're studying and divulging information that you believe is dangerous. A journal of bad results would be incredible. You could have one for novel or interesting failures as the article posits, and you could have one that catalogued and categorized more mundane failures - if its not an i…

The problem with a journal of bad results is that it would only be useful if you kept meticulous notes while getting bad results. Which functionally turns them into good results which can be published normally, since you now have enough content to write something like " not found under " which is basically how you call out the original authors. The reality of when you can't reproduce something is usually some combina…

The issue is that there already is so much being published that it is nearly impossible to keep up with in many fields. Any solution that includes more publications just misses the mark and is detrimental IMO. We will only get to better science if we move to a situation were publishing less is acceptable, because the only way we can detect dishonesty is if scientists actually have enough time to follow up on weird results.

Re: Scientific integrity and the ethics of 'utter honesty'

#24
post #17
post #12

Earlier quoted context omitted.

"Would anyone really care to subscribe to, let alone publish in, the American Journal of Discarded Hypotheses, the Annals of Failed Experiments, or PLOS Dumpster?" I have often half-joked about the Journal of Cautionary Tales, and would genuinely find "This particular study went sideways, and here's why..." quite valuable. And also, allowing researchers to still get "credit" for contributing to the knowledge of the d…

> "This particular study went sideways, and here's why..." You often can't tell if it went sideways because your hypothesis was shite, or your experiment was. And when you can guess that your experiment was, you often don't know why .

Sure, but that is a great reason to publish it. It allows other in the community to pick up on where things left off. Maybe someone else has the missing pieces to the puzzle.

Re: Scientific integrity and the ethics of 'utter honesty'

#25
post #12

I think this article kind of misses the mark? There is a big gulf between reporting other explanations and tests for the phenomena you're studying and divulging information that you believe is dangerous. A journal of bad results would be incredible. You could have one for novel or interesting failures as the article posits, and you could have one that catalogued and categorized more mundane failures - if its not an i…

"Would anyone really care to subscribe to, let alone publish in, the American Journal of Discarded Hypotheses, the Annals of Failed Experiments, or PLOS Dumpster?" I have often half-joked about the Journal of Cautionary Tales, and would genuinely find "This particular study went sideways, and here's why..." quite valuable. And also, allowing researchers to still get "credit" for contributing to the knowledge of the d…

We've talked about founding the Journal of the Null Hypothesis, which publishes well-designed experiments which reveal a particular hypothesis to be wrong or insignificant.

Re: Scientific integrity and the ethics of 'utter honesty'

#26

The best autopsy of Jan Hendrik Schon case is " Plastic Fantastic: How the Biggest Fraud in Physics Shook the Scientific World (2009)", by Eugenie Samuel Reich. It explains a lot of the questions asked in this article, such as "how 16 fraudulent papers could have made it through the peer review process of the two premier science journals in the world". The thing about the Schon case is this: he might have got away wi…

"The thing about the Schon case is this: he might have got away with all..." But could Schon have gotten away with it? I somehow doubt it. I was going to mention the Schon case and you beat me to it. I recall when news of this case broke, it then led me to read some of his papers that contained the fabricated data. I've never fully understood why Schon faked the data or what he stood to gain by doing so. First, I've…

The book is pretty illuminating, but at times is kind of grim reading for anyone with past academic experience. What really sank Schon was the failure to replicate his work, but there were a lot of grad students and postdocs who wasted several years attempting to do so, ending up with nothing to show for it. Even so, there was a reluctance to admit that fraud had taken place. From the book:

Following the exposure of a major scientific fraud, senior scientists sometimes voice concern about the possibility of harm to the public image of their field. News reports and transparent investigations sometimes seem to be making the problem worse. But in practice, the scientists most seriously affected by Hendrik Schon's claims were more affected by the reality of fraud than the bad appearance that followed its exposure.

Leading journals like Science and Nature are also reluctant to withdraw papers for similar image reasons, and Schon was a consummate con artist: "He knew it was important to be friendly towards reviewers, and to thank them generously for their valuable comments. He was not to shy to solicit editors politely for their support." He also would go to senior scientists, ask for their opinion on what kind of data would be expected if his imaginary devices worked, and then fabricate data according to their estimates.

Initially, there was also a major pushback against fraud claims by Bell Labs managers, up to giving poor performance reviews to some of Schon's co-workers who questioned his behavior on the basis of them 'not being good team players'.

So, yes, if people had managed to replicate his work, even partially, he'd likely have gotten away with it, and would now be some top scientist in some prestigious institution.

Re: Scientific integrity and the ethics of 'utter honesty'

#27
post #12

I think this article kind of misses the mark? There is a big gulf between reporting other explanations and tests for the phenomena you're studying and divulging information that you believe is dangerous. A journal of bad results would be incredible. You could have one for novel or interesting failures as the article posits, and you could have one that catalogued and categorized more mundane failures - if its not an i…

"Would anyone really care to subscribe to, let alone publish in, the American Journal of Discarded Hypotheses, the Annals of Failed Experiments, or PLOS Dumpster?" I have often half-joked about the Journal of Cautionary Tales, and would genuinely find "This particular study went sideways, and here's why..." quite valuable. And also, allowing researchers to still get "credit" for contributing to the knowledge of the d…

There's always RISKS digest: https://catless.ncl.ac.uk/Risks/

Re: Scientific integrity and the ethics of 'utter honesty'

#28
> International Journal of Interesting Negative Results

"Interesting" results would presumably include failures to replicate the results of other researchers. Those need to be published.

But really, for an honest scientist, aren't negative results at least as interesting as positive results? I'm assuming that a positive result is one that confirms some theory or assumption; so a negative result is actually more interesting, in that it kicks over the cauldron, and forces a re-evaluation of previous theories and assumptions. A positive result means "All is groovy; rudder amidships, full ahead". A negative result means "Wait - where exactly are we anyway?"

Re: Scientific integrity and the ethics of 'utter honesty'

#29
post #17
post #12

Earlier quoted context omitted.

"Would anyone really care to subscribe to, let alone publish in, the American Journal of Discarded Hypotheses, the Annals of Failed Experiments, or PLOS Dumpster?" I have often half-joked about the Journal of Cautionary Tales, and would genuinely find "This particular study went sideways, and here's why..." quite valuable. And also, allowing researchers to still get "credit" for contributing to the knowledge of the d…

> "This particular study went sideways, and here's why..." You often can't tell if it went sideways because your hypothesis was shite, or your experiment was. And when you can guess that your experiment was, you often don't know why .

But you often can.

All journals are selective. This would select for "We know why, and there's practical knowledge to be gained from the why."

Re: Scientific integrity and the ethics of 'utter honesty'

#30

The best autopsy of Jan Hendrik Schon case is " Plastic Fantastic: How the Biggest Fraud in Physics Shook the Scientific World (2009)", by Eugenie Samuel Reich. It explains a lot of the questions asked in this article, such as "how 16 fraudulent papers could have made it through the peer review process of the two premier science journals in the world". The thing about the Schon case is this: he might have got away wi…

"The thing about the Schon case is this: he might have got away with all..." But could Schon have gotten away with it? I somehow doubt it. I was going to mention the Schon case and you beat me to it. I recall when news of this case broke, it then led me to read some of his papers that contained the fabricated data. I've never fully understood why Schon faked the data or what he stood to gain by doing so. First, I've…

> Sure, by then I had the hindsight of knowing that parts of those papers contained fabricated data but even so it was pretty obvious.

But this IS the crux of the problem. Chess has a similar situation--the forced checkmate is obvious after the fact. However, even chess masters miss forced checkmates up front at the table all the time. And that situation is far less subtle than research data.

> I was absolutely furious and I insisted that he stay back with me in the lab over lunchtime whilst I repeated the titration which he did.

I applaud him, but that's not good lab technique. Good lab technique is "Here's my lab notebook. You can see my data along with the handwritten timestamps. The procedure is also written in there. YOU will replicate this while I watch and correct YOUR lab technique." THAT'S good lab technique.

I had a physics lab as a freshman taught by an absolute tyrant of a professor. He would pull one notebook every lab and try to replicate your results. If he couldn't, you failed that lab. You had to record EVERYTHING in your lab notebook. Serial numbers of equipment, test and calibration procedure steps, data that you knew was invalid and record the reason why ... all in pen with timestamps with no erasures ever allowed.

It was brutal.

He was totally unapologetic. His stance: "If you can't get the same result three times, you don't even know that you're wrong let alone that you're right. I'm actually being generous in the lab--you only have to get the same result twice."

Post reply on HN