Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Most Published Research Findings Are False

ncbi.nlm.nih.gov

Post reply on HN
Print view View original

21–30 of 45 posts

Re: Why Most Published Research Findings Are False

#21

Post by _delirium »

vitaminj wrote: post #20

In statistics, you're supposed to come up with a statistical model first before running regressions on the data. But quite a few papers I've read (especially in finance) seem to go the other way around, i.e. They run regressions on a data set, adding and subtracting independent variables until the t values and standard errors start looking good. Then they construct the linear model, assume the Gauss-Markov assumption…

In practice that's fairly common in all areas of science. You look for patterns in data and infer a relationship/equation/etc. Of course, you are supposed to confirm that it actually holds in new data / subsequent experiments.

Widespread use of data-mining software does make it much easier to do dodgy things on a wide scale.

Re: Why Most Published Research Findings Are False

#22

Post by [deleted] »

ramanujan wrote: post #5

This paper has gotten way too much press for an oversimplified model of science. Here's the thing: if results hold up to scrutiny, the authors are eager to share code and plasmids/samples. If not, they are a lot more squirrelly. Outside replication is what keeps the machine moving forward, is fairly readily proxied by citation rates, and yet is not captured by Ioannidis' simple model.

[deleted]

Re: Why Most Published Research Findings Are False

#23

Post by drewse »

This title yelled "paradox!" at me. It's funny to see it coming from a ".gov" website.

For those who need clarification, if this published research and its title are true, than it is saying that research like itself are usually false. This contradicts the original assumption that it is true.

If this published research and its title are false, than research like itself is usually true since what it's saying must be wrong. This contradicts the original assumption that it is false.

Re: Why Most Published Research Findings Are False

#25

Post by ck2 »

It's because researchers slack just like everyone else at their jobs and need to pay the bills in the meanwhile. Now imagine your doctor or law enforcement and the mess they cause when they slack and cut corners just to produce "product" and justify their jobs.

Re: Why Most Published Research Findings Are False

#26

Post by jforman »

alokm wrote: post #10
elbenshira wrote: post #4

First of all, the author of this piece works in Department of Hygiene and Epidemiology. Research is done differently across different disciplines, so it's dangerous to try to expand this to other disciplines. For example, some fields find alpha But research is very weird indeed. The more conference/journal articles you read, the less you trust them. I mean, say a field accepts results alpha Feel free to correct me if…

You will have to use binomial distribution ( http://en.wikipedia.org/wiki/Binomial_distribution ) for finding the probability that 5% of them are wrong (Assuming researches are independent). I guess this probability will come to be very small. But the point of this article is not alpha(confidence level) ,It is the bias of the researcher.

p-values are calculated in many different ways, not always using the binomial distribution (of the p-values I've claimed, very few have used the binomial distribution)

Re: Why Most Published Research Findings Are False

#27

Post by samd »

ramanujan wrote: post #5

This paper has gotten way too much press for an oversimplified model of science. Here's the thing: if results hold up to scrutiny, the authors are eager to share code and plasmids/samples. If not, they are a lot more squirrelly. Outside replication is what keeps the machine moving forward, is fairly readily proxied by citation rates, and yet is not captured by Ioannidis' simple model.

That's not the case, even the most widely cited research is dubious. From the Atlantic article:

"He zoomed in on 49 of the most highly regarded research findings in medicine over the previous 13 years, as judged by the science community’s two standard measures: the papers had appeared in the journals most widely cited in research articles, and the 49 articles themselves were the most widely cited articles in these journals. These were articles that helped lead to the widespread popularity of treatments such as the use of hormone-replacement therapy for menopausal women, vitamin E to reduce the risk of heart disease, coronary stents to ward off heart attacks, and daily low-dose aspirin to control blood pressure and prevent heart attacks and strokes. Ioannidis was putting his contentions to the test not against run-of-the-mill research, or even merely well-accepted research, but against the absolute tip of the research pyramid. Of the 49 articles, 45 claimed to have uncovered effective interventions. Thirty-four of these claims had been retested, and 14 of these, or 41 percent, had been convincingly shown to be wrong or significantly exaggerated. If between a third and a half of the most acclaimed research in medicine was proving untrustworthy, the scope and impact of the problem were undeniable. That article was published in the Journal of the American Medical Association."

Re: Why Most Published Research Findings Are False

#28

Post by usaar333 »

drewse wrote: post #23

This title yelled "paradox!" at me. It's funny to see it coming from a ".gov" website. For those who need clarification, if this published research and its title are true, than it is saying that research like itself are usually false. This contradicts the original assumption that it is true. If this published research and its title are false, than research like itself is usually true since what it's saying must be wr…

Amusing at first glance, yes.

Of course he's talking about studies that use significance tests, which his own paper isn't directly using to prove his point.

Re: Why Most Published Research Findings Are False

#29

Post by izendejas »

samd wrote: post #27
ramanujan wrote: post #5

This paper has gotten way too much press for an oversimplified model of science. Here's the thing: if results hold up to scrutiny, the authors are eager to share code and plasmids/samples. If not, they are a lot more squirrelly. Outside replication is what keeps the machine moving forward, is fairly readily proxied by citation rates, and yet is not captured by Ioannidis' simple model.

That's not the case, even the most widely cited research is dubious. From the Atlantic article: "He zoomed in on 49 of the most highly regarded research findings in medicine over the previous 13 years, as judged by the science community’s two standard measures: the papers had appeared in the journals most widely cited in research articles, and the 49 articles themselves were the most widely cited articles in these jo…

Were some of those citations made to prove those findings wrong? Dumb question, but would be a shame to not be thourough when attacking bad literature.
Post reply on HN