Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Most Published Research Findings Are False

ncbi.nlm.nih.gov

Post reply on HN
Print view View original

11–20 of 45 posts

Re: Why Most Published Research Findings Are False

#11

Post by greendestiny »

elbenshira wrote: post #4

First of all, the author of this piece works in Department of Hygiene and Epidemiology. Research is done differently across different disciplines, so it's dangerous to try to expand this to other disciplines. For example, some fields find alpha But research is very weird indeed. The more conference/journal articles you read, the less you trust them. I mean, say a field accepts results alpha Feel free to correct me if…

If you only publish results p < 0.05 then you can't say what percentage are due to chance. It could be all of them. All it tells you is how many experiments would get that significance level through chance. To know the number of results that are simply due to that effect you'd have to know the prevalence of actual positive results (ie not due to chance). If a actual positives are common then it could be much lower than 5% reported results due to chance, if actual positives are impossible then it could 100% due to chance.

Re: Why Most Published Research Findings Are False

#16

Post by tokenadult »

dholowiski wrote: post #13
zbanks wrote: post #9

Self reference much? This is published research... Clearly these findings are false... or maybe not? Dammit. http://en.wikipedia.org/wiki/Liar_paradox

I was thinking the same thing. We're sure it's not April 1st?

The submitted article is a review article about methodology, for the most part, and isn't announcing brand-new primary experimental research findings. So the submitted article is distinguishable from the kind of articles it analyzes. See

http://en.wikipedia.org/wiki/Wikipedia:MEDRS

for more on distinctions among differing kinds of publications on research.

Re: Why Most Published Research Findings Are False

#17

Post by cdavid »

elbenshira wrote: post #4

First of all, the author of this piece works in Department of Hygiene and Epidemiology. Research is done differently across different disciplines, so it's dangerous to try to expand this to other disciplines. For example, some fields find alpha But research is very weird indeed. The more conference/journal articles you read, the less you trust them. I mean, say a field accepts results alpha Feel free to correct me if…

Actually, having an alpha of value x does NOT mean that 100 * x % are false. It only gives you an indication of the coverage of your experiment, which is useful when compared to other, independent studies.

I think most scientists don't understand the meaning of the p-value. There was an interesting discussion last year in the statistical blog community on that question, with leading statisticians involved in it: http://radfordneal.wordpress.com/2009/03/07/does-coverage-ma...

Re: Why Most Published Research Findings Are False

#18

Post by zeteo »

elbenshira wrote: post #4

First of all, the author of this piece works in Department of Hygiene and Epidemiology. Research is done differently across different disciplines, so it's dangerous to try to expand this to other disciplines. For example, some fields find alpha But research is very weird indeed. The more conference/journal articles you read, the less you trust them. I mean, say a field accepts results alpha Feel free to correct me if…

"I mean, say a field accepts results alpha And then there are fields like climate science, where "very high confidence" means 10% probability of being wrong:

http://ipccinfo.com/

Re: Why Most Published Research Findings Are False

#20

Post by vitaminj »

In statistics, you're supposed to come up with a statistical model first before running regressions on the data. But quite a few papers I've read (especially in finance) seem to go the other way around, i.e.

They run regressions on a data set, adding and subtracting independent variables until the t values and standard errors start looking good.

Then they construct the linear model, assume the Gauss-Markov assumptions and sometimes (though not always) try to explain the causal relationship between the variables.

This is obviously very wrong and nobody has any clue what the distribution of the least squares estimators to these models are. But I've seen plenty of examples of this, which is enough to void the results of the paper (even if the model they come up with is somewhat plausible).

Post reply on HN