Why most published scientific research is probably false [video]
1–10 of 57 posts
Re: Why most published scientific research is probably false [video]
#2Probably false? As in you have a better chance claiming the negation of a scientific paper's conclusion than the actual conclusion? I doubt it.
Re: Why most published scientific research is probably false [video]
#3A good opportunity for mentioning Benford's Law: http://en.wikipedia.org/wiki/Benford%27s_law#Scientific_frau...
Re: Why most published scientific research is probably false [video]
#4Probably false? As in you have a better chance claiming the negation of a scientific paper's conclusion than the actual conclusion? I doubt it.
[deleted]
Re: Why most published scientific research is probably false [video]
#5so... non-scientific articles about scientific research are true? probably not.
Re: Why most published scientific research is probably false [video]
#6Probably false? As in you have a better chance claiming the negation of a scientific paper's conclusion than the actual conclusion? I doubt it.
yes, probably false. This is not news, e.g. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1182327/ came out in 2005.
of course, research "proving" that research tends to be false can get us into a little circular game...
Re: Why most published scientific research is probably false [video]
#7Probably false? As in you have a better chance claiming the negation of a scientific paper's conclusion than the actual conclusion? I doubt it.
In certain fields, that's not implausible. Many results are along the lines of "compound X is effective against disease Y", where the negation, "compound X is not effective against disease Y", is a reasonable baseline assumption, because most compounds are not effective against most diseases.
Results where the prior odds could plausibly be considered 50/50 are another story. Scientific research that doesn't take the form "rule out the null hypothesis with p values" is also a different matter (lots of CS and physics, among other fields, has a more complex mingling of theory and epistemology than experiment/nullhypothesis/pvalue/repeat).
Re: Why most published scientific research is probably false [video]
#8Probably false? As in you have a better chance claiming the negation of a scientific paper's conclusion than the actual conclusion? I doubt it.
I wouldn't be so sure. Not talking about mathematics or CS, but in many social sciences (and also medicine and so on) the paper is just stating that A->B but i) there can be a lot more things going on that explain whatever correlation you are finding (from reverse causality to bad experimental setup), and *more importantly ii) in order to get published, you need to present a somewhat interesting or controversial statement.
If you state something obvious and most likely true, good luck getting to Nature/Science. If you state something unusual, then that will sell (where selling is getting citations) so you will see it published
Re: Why most published scientific research is probably false [video]
#9Probably false? As in you have a better chance claiming the negation of a scientific paper's conclusion than the actual conclusion? I doubt it.
yes, probably false. This is not news, e.g. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1182327/ came out in 2005. of course, research "proving" that research tends to be false can get us into a little circular game...
There's a follow-up paper to the one you linked that claims that much less published research findings are false:
>We estimate that the overall rate of false discoveries among reported results is 14% (s.d. 1%), contrary to previous claims. We also found that there is no a significant increase in the estimated rate of reported false discovery results over time (0.5% more false positives (FP) per year, P=0.18) or with respect to journal submissions (0.5% more FP per 100 submissions, P=0.12).
[1] http://biostatistics.oxfordjournals.org/content/early/2013/0...
The circle has begun :)
Re: Why most published scientific research is probably false [video]
#10Look. Clearly science makes progress that's "true" in the sense that it becomes useful and can be used as functional models of the way things work.
This video's using a very reductionist kind of statistics to point out that, yes, an individual piece of research making a claim might have a good chance of being wrong. Which is why science doesn't say, "Oh, well Larry just proved that Saturn orbits Uranus so let's just never think about that again and instead move on to proving that the Sun is fully powered by excess heat radiating off of Elon Musk's brain." Science is a process that works in aggregate, using a large volume of research and scientists checking one another as a way to smooth over this very imperfect process. Over time. Science checks itself. That's the whole point. That's why it reaches some pretty damned good conclusions about the way things work.
So.
I don't know what the point of this video is. Science is wrong? Scientists are stupid? The Economist is smart? I should believe the Republicans when they say the Earth couldn't possibly be warming because that one time it snowed in a part of Texas where it never really snows all that often?