You don't even need the graphs, miles driven per year is a sum. Sums converge to gaussians yet here the deviation was half of mean which means they used a uniform variable generator for miles.
Taleb pointed this out on twitter iirc
Noted study in psychology fails to replicate, crumbles with evidence of fraud
21–30 of 110 posts
Re: Noted study in psychology fails to replicate, crumbles with evidence of fraud
#22Earlier quoted context omitted.
Failure to replicate is the definition of bad science. Science is based on objective, repeatable experiments. If the experiments can't even be reproduced how can they be hypothesized and validated?
If we report p values around 0.05 wouldn’t you expect about 1 in 20 studies failing to replicate without being bad science?
[deleted]
Re: Noted study in psychology fails to replicate, crumbles with evidence of fraud
#23What's the purpose of this? It will just give ammo to anti-science Trump like people. It's harmful. This kind of stuff should be handled as quietly as possible, and definitely not in public. Science is already under attack, we need not undermine it from the inside too.
Not sure if serious, but I disagree. I think scientists playing games with public perception is a big part of what undermined their credibility in the first place. Also this article talks about Ariely loudly promoting his own work after claiming he already should have known it failed to replicate. If scientific studies are being used to drive public discourse, then debunking them should be public discourse too. The very strength of science is its ability to be self-critical. The mechanics of that shouldn't be hidden away.
Re: Noted study in psychology fails to replicate, crumbles with evidence of fraud
#24In private I have heard of science done that is suspected to be fraudulent. The results are so intriguing and promising as to send multiple grad students across multiple groups barreling into replicating and expanding the results. What happened? Years of time wasted with no publications. PIs and journals aren't interested in publishing negative results. The person who lied in their research is reaping career benefits…
Publishing negative results is important to solving this issue.
How about instead of applying to journals with a finished study, people would apply with a proposal of a study and then publish the results independent of how spectacular/boring they turn out?
Journals would then have to select studies based on their design and based on the question and scientists could just do science.
Re: Noted study in psychology fails to replicate, crumbles with evidence of fraud
#25Earlier quoted context omitted.
Failure to replicate is the definition of bad science. Science is based on objective, repeatable experiments. If the experiments can't even be reproduced how can they be hypothesized and validated?
If we report p values around 0.05 wouldn’t you expect about 1 in 20 studies failing to replicate without being bad science?
In the social sciences? If they're reproducing the earlier results with the original data and code, the results should be exactly the same.
If they're replicating with a follow-on study of some kind, it depends on any number of factors, since it's almost impossible to do a follow-on survey-based or lab experiment or use a similar or newer dataset while accounting for and controlling exactly the same set of conditions (this is one of the issues that I've heard used to distinguish between hard and soft sciences).
E.g., if paper A has an observation count of 200 (insert your preferred number), a replication study could hand out exactly the same survey to another 200 respondents and get similar results that, nonetheless, fail to exactly replicate the earlier findings (or, over a small number of samples, fail to conform to a normal distribution).
This doesn't mean that the original study was "bad science" or that the social sciences in general are "bad", only that we have to temper our expectations and maintain a healthy dose of skepticism.
Re: Noted study in psychology fails to replicate, crumbles with evidence of fraud
#26What's the purpose of this? It will just give ammo to anti-science Trump like people. It's harmful. This kind of stuff should be handled as quietly as possible, and definitely not in public. Science is already under attack, we need not undermine it from the inside too.
Really? Covering up a messy truth because you perceive it as not politically expedient strikes me as the exact opposite of science. That road leads to worse knowledge, not better.
Re: Noted study in psychology fails to replicate, crumbles with evidence of fraud
#27Earlier quoted context omitted.
Publishing negative results is important to solving this issue.
How about instead of applying to journals with a finished study, people would apply with a proposal of a study and then publish the results independent of how spectacular/boring they turn out? Journals would then have to select studies based on their design and based on the question and scientists could just do science.
This is a good idea, and is known as "preregistration". https://en.wikipedia.org/wiki/Preregistration_(science)
Hopefully it will become more common practice.
Re: Noted study in psychology fails to replicate, crumbles with evidence of fraud
#28What's the purpose of this? It will just give ammo to anti-science Trump like people. It's harmful. This kind of stuff should be handled as quietly as possible, and definitely not in public. Science is already under attack, we need not undermine it from the inside too.
Anti-science people tend to be conspiracy-minded and believe that the establishment is a homogeneous group that is trying to mislead the public.
Creating an actual conspiracy to mislead the public is not a good way to fight this perception.
Re: Noted study in psychology fails to replicate, crumbles with evidence of fraud
#29What's the purpose of this? It will just give ammo to anti-science Trump like people. It's harmful. This kind of stuff should be handled as quietly as possible, and definitely not in public. Science is already under attack, we need not undermine it from the inside too.
I disagree. Constant re-evaluation of evidence and improvement of working theories are essential to science. Science needs transparency and honest critique.
Acknowledging falsified results is not an attack on science, it is how science works. It shows that science is not a mysterious cult of knowledge conspiring to hide information and deceive others, but rather the collective effort of imperfect humans who want to understand the universe and make things better for us all.
Re: Noted study in psychology fails to replicate, crumbles with evidence of fraud
#30What's the purpose of this? It will just give ammo to anti-science Trump like people. It's harmful. This kind of stuff should be handled as quietly as possible, and definitely not in public. Science is already under attack, we need not undermine it from the inside too.
What you just said goes against everything science stands for, otherwise it just becomes a belief system, and you're no better than what you warn against.