Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why most published scientific research is probably false [video]

economist.com

Post reply on HN
Print view View original

21–30 of 57 posts

Re: Why most published scientific research is probably false [video]

#21

Post by epistasis »

pygy_ wrote: post #17
chasing wrote: post #10

Look. Clearly science makes progress that's "true" in the sense that it becomes useful and can be used as functional models of the way things work. This video's using a very reductionist kind of statistics to point out that, yes, an individual piece of research making a claim might have a good chance of being wrong. Which is why science doesn't say, "Oh, well Larry just proved that Saturn orbits Uranus so let's just…

> Science checks itself. Not that much, actually. There's no point in reproducing the work of others to advance your own scientific career. Unless your research directly directly depends on them, it is counterproductive.

In the important areas, it does check itself. If a scientist has been doing some interesting work, she will soon find herself with colleagues working on the same area. If she has not been doing something interesting, that line of research does not get cited, dies out, and does not get funding.

In active learning, there's a tradeoff between exploration and exploitation. Science does this too, but heuristically.

Re: Why most published scientific research is probably false [video]

#23

Post by [deleted] »

chasing wrote: post #10

Look. Clearly science makes progress that's "true" in the sense that it becomes useful and can be used as functional models of the way things work. This video's using a very reductionist kind of statistics to point out that, yes, an individual piece of research making a claim might have a good chance of being wrong. Which is why science doesn't say, "Oh, well Larry just proved that Saturn orbits Uranus so let's just…

[deleted]

Re: Why most published scientific research is probably false [video]

#24

Post by pallandt »

kineticfocus wrote: post #18
pallandt wrote: post #3

A good opportunity for mentioning Benford's Law: http://en.wikipedia.org/wiki/Benford%27s_law#Scientific_frau...

Maybe an opportunity for mentioning Simpson's Paradox: http://en.wikipedia.org/wiki/Simpson%27s_paradox

Good point!

Re: Why most published scientific research is probably false [video]

#25

Post by gabriel34 »

Here is the link to the source, much friendlier to HN folk that rather read than watch: http://www.economist.com/news/briefing/21588057-scientists-t...

EDIT: To add on to this, the source to the video does make an interesting observation about statistical methods being employed by scientists who doesn't know their pitfalls.

Other thing I take from a more careful reading (as opposed to viewing a two minute highly superficial video) is that the article makes the assumption that all hypothesis will be subject to only one study, if we have three studies denying a certain hypothesis and one confirming it it's pretty easy to catch the false positive on a literature review article (routinely done by people entering academia)

Re: Why most published scientific research is probably false [video]

#26

Post by gabriel34 »

pygy_ wrote: post #17
chasing wrote: post #10

Look. Clearly science makes progress that's "true" in the sense that it becomes useful and can be used as functional models of the way things work. This video's using a very reductionist kind of statistics to point out that, yes, an individual piece of research making a claim might have a good chance of being wrong. Which is why science doesn't say, "Oh, well Larry just proved that Saturn orbits Uranus so let's just…

> Science checks itself. Not that much, actually. There's no point in reproducing the work of others to advance your own scientific career. Unless your research directly directly depends on them, it is counterproductive.

Actually there are loads of scientific papers only reviewing other papers (usually a set of them), trying to infer conclusions and criticizing them on methodology and rigidity. Scientific careers advance both in quality and in quantity of papers published. Not everyone gets featured on Science or Nature.

Re: Why most published scientific research is probably false [video]

#27

Post by gabriel34 »

lvs wrote: post #19

This is extremely misleading and feeds an anti-intellectual notion that scientists are just lying to everybody. First, it perpetuates a common claim of those who don't practice any sort of science: that the output of scientific studies is an enumeration of true/false claims determined with statistical inference logic. (Science media/blogs really aren't helping on this one.) Second, the math is just wrong: the space o…

Isn't the space of already enumerated and studied hypotheses finite?

Re: Why most published scientific research is probably false [video]

#30

Post by 001sky »

epistasis wrote: post #21
pygy_ wrote: post #17

Earlier quoted context omitted.

> Science checks itself. Not that much, actually. There's no point in reproducing the work of others to advance your own scientific career. Unless your research directly directly depends on them, it is counterproductive.

In the important areas, it does check itself. If a scientist has been doing some interesting work, she will soon find herself with colleagues working on the same area. If she has not been doing something interesting, that line of research does not get cited, dies out, and does not get funding. In active learning, there's a tradeoff between exploration and exploitation. Science does this too, but heuristically.

If she has not been doing something interesting, that line of research does not get cited, dies out, and does not get funding.

Could you explain how this verifies results? Citation (alone) isn't verification. And its safe to say followers and funding correlate with one another, at least statistically.

Post reply on HN