I love polymarket. Prediction markets on topics like this are about the only way to discern truth from afar.
Noted study in psychology fails to replicate, crumbles with evidence of fraud
11–20 of 110 posts
Re: Noted study in psychology fails to replicate, crumbles with evidence of fraud
#12I'm pretty sure this was discussed in an earlier HN post. However, I want to point out that "not replicating" doesn't necessarily equate to bad science (or, more accurately, bad practice on the part of the principal investigators irt accepted best practices in their field). There are a bunch of studies in the social sciences that failed to replicate simply because a p value was slightly below a threshold target, even…
Failure to replicate is the definition of bad science. Science is based on objective, repeatable experiments. If the experiments can't even be reproduced how can they be hypothesized and validated?
Being possible to replicate isn’t the same as every experiment getting identical results. The point of doing statistics is to separate signal from noise, but it also means ignoring results where the signal wasn’t strong enough because your sample size was to small etc.
Suppose your testing a weight loss drug and in the first experiment subjects lost 5 pounds and the second they only lost 3. On the surface the first study replicates fine, except the second study has a weaker signal so it might not past the statistical threshold chosen before the study. This is fairly common issue as you should expect random noise to amplify a signal in about half of experiments and more than half of all published research.
Re: Noted study in psychology fails to replicate, crumbles with evidence of fraud
#13In private I have heard of science done that is suspected to be fraudulent. The results are so intriguing and promising as to send multiple grad students across multiple groups barreling into replicating and expanding the results. What happened? Years of time wasted with no publications. PIs and journals aren't interested in publishing negative results. The person who lied in their research is reaping career benefits…
Publishing negative results is important to solving this issue.
Re: Noted study in psychology fails to replicate, crumbles with evidence of fraud
#14I have several books of Ariely. I enjoy reading psychology, but I stopped reading pop psychology books and go for academic books or textbooks or individual articles.
The amount of times I read about spectacular counter-intuitive study that "changes everything" and later find out it was fraudulent or failed to replicate makes me really doubt all of my knowledge I have from reading about psychology.
Reproducibility crisis is real and it sucks for all the people that read individual books and don't know about reproducibility. I feel people that write those books should also put themself into higher standards and stop writing narrative books based on individual nonreplicated studies.
You can read about reproducibility crisis here: https://www.theatlantic.com/science/archive/2018/11/psycholo...
Re: Noted study in psychology fails to replicate, crumbles with evidence of fraud
#15In private I have heard of science done that is suspected to be fraudulent. The results are so intriguing and promising as to send multiple grad students across multiple groups barreling into replicating and expanding the results. What happened? Years of time wasted with no publications. PIs and journals aren't interested in publishing negative results. The person who lied in their research is reaping career benefits…
Peer review, as we know it today, is a fairly new phenomenon and most of our best science was done before it entered the academy. It is a pox on innovation.
Re: Noted study in psychology fails to replicate, crumbles with evidence of fraud
#16In private I have heard of science done that is suspected to be fraudulent. The results are so intriguing and promising as to send multiple grad students across multiple groups barreling into replicating and expanding the results. What happened? Years of time wasted with no publications. PIs and journals aren't interested in publishing negative results. The person who lied in their research is reaping career benefits…
Publishing negative results is important to solving this issue.
I agree. The grad students agree. Now how do you convince a group obsessed with career advancement and holding all of the cards (PIs) to agree?
Re: Noted study in psychology fails to replicate, crumbles with evidence of fraud
#17In private I have heard of science done that is suspected to be fraudulent. The results are so intriguing and promising as to send multiple grad students across multiple groups barreling into replicating and expanding the results. What happened? Years of time wasted with no publications. PIs and journals aren't interested in publishing negative results. The person who lied in their research is reaping career benefits…
The fact that the "negative" results don't get published is the worst part. Failing to replicate a strong effect should be a result in and of itself! Failing to find support for a plausible theory is also a result. The fact that this kind of publication bias still exists doesn't make any sense to me. And it seems to be one of the primary ills of scientific publishing and academia.
Re: Noted study in psychology fails to replicate, crumbles with evidence of fraud
#18My bet would be that a surprisingly large number of psychology studies are completely irreproducible. I'm even skeptical of a lot of fMRI experiments. Meta-analyses certainly can give a bit better insight, but have their own problems.
Re: Noted study in psychology fails to replicate, crumbles with evidence of fraud
#19You don't even need the graphs, miles driven per year is a sum. Sums converge to gaussians yet here the deviation was half of mean which means they used a uniform variable generator for miles.
[deleted]
Re: Noted study in psychology fails to replicate, crumbles with evidence of fraud
#20What's the purpose of this? It will just give ammo to anti-science Trump like people.
It's harmful. This kind of stuff should be handled as quietly as possible, and definitely not in public.
Science is already under attack, we need not undermine it from the inside too.