Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Evidence of Fraud in an Influential Study About Procrastination

datacolada.org

Post reply on HN
Print view View original

141–150 of 273 posts

Re: Evidence of Fraud in an Influential Study About Procrastination

#141

Post by dilawar »

jarenmf wrote: post #133

Science is self-correcting, but sometimes, the correction mechanism it is very slow and inconsequential enough that people are incentivized to just publish anything. Peer review is merely a quality check—and not a very good one. Just the other day, I ran into a paper in my field that claims to have solved a central problem that scientists have been working on for 50 years, single-authored, using some fancy mathematic…

> Science is self-correcting, but sometimes

As long as people mislead or make mistake unknowingly, it's fine to rely to self-correcting mechanism of science. The institutional rot is much deeper. It is not good to leave all this fraud to self-correcting mechanisms of science. We also have "trust" to worry about.

Re: Evidence of Fraud in an Influential Study About Procrastination

#143

Post by stackghost »

jarenmf wrote: post #133

Science is self-correcting, but sometimes, the correction mechanism it is very slow and inconsequential enough that people are incentivized to just publish anything. Peer review is merely a quality check—and not a very good one. Just the other day, I ran into a paper in my field that claims to have solved a central problem that scientists have been working on for 50 years, single-authored, using some fancy mathematic…

>I made a promise to myself that if no real action is taken, I will seriously consider changing careers.

Personally I think you should name and shame the author as well as the publications and the names of the peer "reviewers" who clearly did sweet-fuck-all.

Re: Evidence of Fraud in an Influential Study About Procrastination

#144

Post by yayamo »

recursivecaveat wrote: post #110
mulanroo wrote: post #87

One thing I didn't know before reading this: an unusually large effect size can itself be a red flag. You don't have to understand all the statistics to at least notice when a result looks almost too good to be true.

It is interesting how all these famous non-replicating psych paper results feel reasonably intuitive to me. Like I can buy that having a more structured and detailed deadline helps you procrastinate less. I can even buy that doing a goofy little "power pose" exercise kind of hypes yourself up to give some self confidence and do better on an interview or whatever. You would almost expect the opposite and that all the…

Your exact intuition about the implausibility of many large effect sizes is called the “Piranha problem” by my favourite statistician Andrew Gelman first here in a blog post: https://statmodeling.stat.columbia.edu/2017/12/15/eats/

And then as a published paper: https://sites.stat.columbia.edu/gelman/research/published/pi...

> how many explanatory variables can have large effects on an outcome?

Re: Evidence of Fraud in an Influential Study About Procrastination

#145

Post by rvba »

jarenmf wrote: post #133

Science is self-correcting, but sometimes, the correction mechanism it is very slow and inconsequential enough that people are incentivized to just publish anything. Peer review is merely a quality check—and not a very good one. Just the other day, I ran into a paper in my field that claims to have solved a central problem that scientists have been working on for 50 years, single-authored, using some fancy mathematic…

Would be nice if you kepy ypur career and created some magazine / journal that focuses on checks and pointing out flaws.

That cheaters would dread being included there (would probably be a thick journal too)

Re: Evidence of Fraud in an Influential Study About Procrastination

#146

Post by crimsoneer »

At the risk of bringing up a controversial topic, in the midst of the Jason Arday furore last month when everyone was like "this wouldn't happen to a white professor", the person who kept coming to mind for me was Ariely. I became a data scientist in no small part due to his early work, and seeing academia totally incapable of punishing him is deeply disappointing.

Re: Evidence of Fraud in an Influential Study About Procrastination

#147

Post by mike_hearn »

jarenmf wrote: post #133

Science is self-correcting, but sometimes, the correction mechanism it is very slow and inconsequential enough that people are incentivized to just publish anything. Peer review is merely a quality check—and not a very good one. Just the other day, I ran into a paper in my field that claims to have solved a central problem that scientists have been working on for 50 years, single-authored, using some fancy mathematic…

Science isn't self correcting. This is a harmful meme that we need to collectively put in the trash can, as it's constantly used by academics to evade accountability.

The core problem is people conflating science with Science™. The scientific method is a set of techniques that help people discover and report true things. But there are two huge problems with it:

1. What exactly is the scientific method? It's not like there's a constitutional amendment that lays it out. Try and pin people down and you'll discover nobody agrees on what science actually requires.

2. Even if some subset of people did write down a definition, nobody is checking anyone is following it.

So people conflate science the Platonic methodological ideal with Science™, the institutions that claim most forcefully to be its avatar. But those institutions have neither a definition nor any enforcement arm, so fraud and abuse runs rampant. There's no reason why anything should self correct in this environment outside of corporate science, where the self-correction mechanism is managerial intervention or, in the absence of that, bankruptcy.

Re: Evidence of Fraud in an Influential Study About Procrastination

#148

Post by coldtea »

inigyou wrote: post #131
coldtea wrote: post #129

Earlier quoted context omitted.

If it gets so bad that a society/academic system needs to criminalize such fraud, they have already lost that race. The only thing that really works is a culture where this rarely if ever happens, because of the peer respect and self-respect of academics.

[flagged]

>Do you know a trick to pay my bills in self-respect? Most of my creditors demand I pay in money.

Nobody said that shit. You can pay your builds with the monetary salary from doing your job honestly. That was my first point.

If many people discover that such a salary is not enough to get by, or if many people aren't content with "enough to get by" and decide to perform academic fraud for extras, then that's a society that already lost the plot. That was my second point.

Re: Evidence of Fraud in an Influential Study About Procrastination

#149

Post by zhivota »

I hope that with LLMs we'll see a lot faster identification of these anomalies. Imagine that this guy taught at MIT and has his own lab at Duke, basically more than 20 years of taking up some of the most prestigious positions in his field, all based on fraud.

The butterfly effect this has on science as a practice is huge. Some marginal researcher who actually deserved it was bumped out of those places, and somewhere down the line someone was bumped all the way out of academia because of this guy's lies. And this is hardly the only person in this situation.

What's scary is that the next fraudster will read this take down and see what they have to do to avoid detection. It wouldn't be that hard to form data that looks a lot more plausible. Replication may be the only solution, unfortunately it's so expensive and time consuming, without any of the actual benefits of doing the original study. Maybe it should be government funded, the US replication lab, its only mission to replicate the top X papers from some named fields each year.

Re: Evidence of Fraud in an Influential Study About Procrastination

#150

Post by mike_hearn »

rvba wrote: post #145
jarenmf wrote: post #133

Science is self-correcting, but sometimes, the correction mechanism it is very slow and inconsequential enough that people are incentivized to just publish anything. Peer review is merely a quality check—and not a very good one. Just the other day, I ran into a paper in my field that claims to have solved a central problem that scientists have been working on for 50 years, single-authored, using some fancy mathematic…

Would be nice if you kepy ypur career and created some magazine / journal that focuses on checks and pointing out flaws. That cheaters would dread being included there (would probably be a thick journal too)

They wouldn't dread being included there, not at all. The institutions don't care about fraud and reflexively defend it, ignore it or shoot the messenger. Look at the Arday nonsense. People are still discovering new lies he was telling but Cambridge went to the mat for him.

Dan Ariely in particular has faced convincing fraud accusations for years:

https://en.wikipedia.org/wiki/Dan_Ariely#Controversies_and_c...

Nothing ever happens. Universities do not care.

Post reply on HN