Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Evidence of Fraud in an Influential Study About Procrastination

datacolada.org

Post reply on HN
Print view View original

171–180 of 273 posts

Re: Evidence of Fraud in an Influential Study About Procrastination

#171

Post by suddenlybananas »

dash2 wrote: post #45
ProjectArcturis wrote: post #5

Dan Ariely remaining at Duke, after it has become clear that much if not most of his work is fraudulent, is at least as big a black mark for Duke as Jason Arday was for Cambridge. Not to mention the Epstein ties.

It's arguably worse. Jason Arday was doing qualitative work in education. Nobody pays any attention to anything UK educationalists write. Dan Ariely was a genuine superstar and writer of popular bestsellers (with academic content).

I agree Ariely was more influential and his fraud more flagrant (obviously Arday's lies were more absurd, but they were less academically relevant), but it's not true that Arday had no attention given to him. He did provide advice to government groups on policy [1,2]. Furthermore, the high publicity around his story about being non-verbal until he was 11 probably gave a lot of false hope to people with non-verbal children.

[1] https://committees.parliament.uk/oralevidence/2482/html/

[2] https://faculty.educ.cam.ac.uk/230223-jason-arday

Re: Evidence of Fraud in an Influential Study About Procrastination

#172

Post by coldtea »

daymanstep wrote: post #156
coldtea wrote: post #129

Earlier quoted context omitted.

If it gets so bad that a society/academic system needs to criminalize such fraud, they have already lost that race. The only thing that really works is a culture where this rarely if ever happens, because of the peer respect and self-respect of academics.

Peer reputation and self policing works in a system where no money is at stake. In the old days scientific research was only done by self-funded rich people with tons of money and free time. Nowadays science is mostly done by people whose income and food depends on getting grants. When your career and income are at stake, why would you take the risk of opposing Dan Ariely? Only someone who was already rich could do t…

>Peer reputation and self policing works in a system where no money is at stake.

It works in systems where money is at stake too. You need a high trust, conscientous society. Doesn't mean that having one solves the issue entirely. The ocassional bad apple will fraud there too, but it wont be rampant enough to be a systemic problem.

>When your career and income are at stake, why would you take the risk of opposing Dan Ariely? Only someone who was already rich could do that.

You don't need independently wealthy scientists to expose fraud.

You just need an environment where if regular salary dependent scientists expose fraud, they're not punished for it.

If you don't have that, sure, then you need independently wealthy scientists to dare.

Re: Evidence of Fraud in an Influential Study About Procrastination

#173

Post by fercircularbuf »

pinkmuffinere wrote: post #105
dbrans wrote: post #37

Earlier quoted context omitted.

Replication is unpaid QA on someone else's product. Nobody gets tenure for finding the bug, hence the field has been meaning to get around to it for 20 years.

On the other hand, this seems like a super interesting path for an (unglamorous) PhD -- replicate the N most important findings in a field, subject to some restrictions (maybe focus on the ones that are "easy", or haven't been replicated yet, or the ones that could be patched together to make an acceptable PhD thesis). This sort of stuff really appeals to me, I think I might end up doing something like that in 5-10 y…

I share your enthusiasm for this. Creating a map of the most influential research across all the fields to identify the highest priority work, any work replicating that work or validating that work by building from it would be a good start. Ideally funded by governments, but institutions, companies, and individuals could put up bounties for replicating particular research.

Re: Evidence of Fraud in an Influential Study About Procrastination

#175

Post by sethammons »

freediddy wrote: post #39
Polizeiposaune wrote: post #32

Earlier quoted context omitted.

That rule would make it difficult to report on a failure to replicate a published paper.

Every paper needs to include an independent replication of the study.

So build a second particle collider to validate what comes out of CERN?

Re: Evidence of Fraud in an Influential Study About Procrastination

#176

Post by tomnicholas1 »

mike_hearn wrote: post #165
aktenlage wrote: post #159

Earlier quoted context omitted.

My take on the self correction is, that later research usually cannot be build on wrong results. So over the decades or centuries the correct results prevail.

You'd think so. I used to believe this too. Surely you try and build on a result which creates an implicit replication study, then your experiments don't work and the problem is discovered pretty quick. Hence, self correction. It often doesn't happen. The reasons seem to vary by field. We do see it in a few very hard fields like materials science. In the softer end of social sciences like anthropology, education or t…

Whilst I weakly agree with this characterisation, it's not at all fair or accurate to put epidemiology and climatology in the same category as your other examples. Yes they have some weaknesses in replication practices (especially for purely computational work), yes the feedback loop for self-correction can be slow, but it would be a mistake to conclude or imply that the major results of those fields are therefore unreliable.

That's because those fields do still make objective empirical predictions about the world, which might take years to be testable but are still ultimately testable. For example, it's been convincingly shown that climate models from decades ago predicted our current climate pretty well. [0]

These fields are more comparable to subfields of physics which only get to do big experiments a handful of times, e.g. because they can only observe so many planets/supernovae/universes, or because they have to spend decades building a new billion-dollar machine to test new hypotheses.

We should also not confuse these arguments with uselessness - soft social science work can still be societally valuable without being easily empirically testable.

[0] https://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/201...

Re: Evidence of Fraud in an Influential Study About Procrastination

#177

Post by enaaem »

mike_hearn wrote: post #165
aktenlage wrote: post #159

Earlier quoted context omitted.

My take on the self correction is, that later research usually cannot be build on wrong results. So over the decades or centuries the correct results prevail.

You'd think so. I used to believe this too. Surely you try and build on a result which creates an implicit replication study, then your experiments don't work and the problem is discovered pretty quick. Hence, self correction. It often doesn't happen. The reasons seem to vary by field. We do see it in a few very hard fields like materials science. In the softer end of social sciences like anthropology, education or t…

The situation of [2] is crazy. Because both the author of [2] and the people using Ferguson’s model are wrong. And the author's advice in one of the last paragraphs of we just need better programmers is also wrong.

The issue is that pandemics are extremely fat-tailed in terms of fatalities [3]. You need to collect a shit tons of data to estimate any reliable parameters and no one has time for that. The only thing you need to know is: Is this a super contagious deadly disease? If yes, lock everything down! This is pretty much what they did in Asia, and countries like Vietnam had zero cases for months and was open while the rest of the world was closed.

How to decide a disease is deadly and super contagious? You need disease experts (who most likely also academics) who understand the nature of the disease, and not better modellers and programmers as suggested in [2]. The issue is that western countries will never let any expert to make that call, so a lot of convincing needs to be done through complicated modelling and real life suffering.

[3] https://arxiv.org/pdf/2004.08658

Re: Evidence of Fraud in an Influential Study About Procrastination

#178

Post by anon-3988 »

mike_hearn wrote: post #147
jarenmf wrote: post #133

Science is self-correcting, but sometimes, the correction mechanism it is very slow and inconsequential enough that people are incentivized to just publish anything. Peer review is merely a quality check—and not a very good one. Just the other day, I ran into a paper in my field that claims to have solved a central problem that scientists have been working on for 50 years, single-authored, using some fancy mathematic…

Science isn't self correcting. This is a harmful meme that we need to collectively put in the trash can, as it's constantly used by academics to evade accountability. The core problem is people conflating science with Science™. The scientific method is a set of techniques that help people discover and report true things. But there are two huge problems with it: 1. What exactly is the scientific method? It's not like…

> Science isn't self correcting....

I am not sure what your proposal is? It seems to be the solution is only more science? This meme that science can't solve anything or is ineffective is also a meme that needs to die. Science is about the only thing that we have to make sense of the world. What else is there?

> 1. What exactly is the scientific method? It's not like there's a constitutional amendment that lays it out. Try and pin people down and you'll discover nobody agrees on what science actually requires.

I am pretty sure the only necessary requirement is repeatability. As far as science is concerned, it does not matter if Jesus did in fact born from a virgin mother, it is not repeatable, and therefore not scientific. We could be entirely wrong about gravity, maybe Earth is indeed only 5 years old and God just created everything to appear the way it is.

Re: Evidence of Fraud in an Influential Study About Procrastination

#179

Post by esalman »

anon-3988 wrote: post #178
mike_hearn wrote: post #147

Earlier quoted context omitted.

Science isn't self correcting. This is a harmful meme that we need to collectively put in the trash can, as it's constantly used by academics to evade accountability. The core problem is people conflating science with Science™. The scientific method is a set of techniques that help people discover and report true things. But there are two huge problems with it: 1. What exactly is the scientific method? It's not like…

> Science isn't self correcting.... I am not sure what your proposal is? It seems to be the solution is only more science? This meme that science can't solve anything or is ineffective is also a meme that needs to die. Science is about the only thing that we have to make sense of the world. What else is there? > 1. What exactly is the scientific method? It's not like there's a constitutional amendment that lays it ou…

> it does not matter if Jesus did in fact born from a virgin mother, it is not repeatable, and therefore not scientific

https://www.newscientist.com/article/1837276-the-boy-whose-b...

Re: Evidence of Fraud in an Influential Study About Procrastination

#180

Post by LeChuck »

It's shocking to me how badly the data was falsified. You'd think a researcher who is A. well versed in statistics and B. has seen a lot of actual research data would be able to produce more believable data. Does this mean we're only seeing the tip of the iceberg? how many papers are based on hard to detect falsifications?
Post reply on HN