Live data from Hacker News

Chart of the Decade: Why You Shouldn’t Trust Every Scientific Study You See

motherjones.com

101–102 of 102 posts

Re: Chart of the Decade: Why You Shouldn’t Trust Every Scientific Study You See

#101
post #76

I often wonder about the kind of research done in computer science--how much is it influenced by the kinds of things that easily get you a Ph.D., versus the kinds of things that are useful but less apparently flashy. A lot of the PL students at my school are extremely wary of doing any follow-up work on ideas that have been published before, even if the implementations of those things are obviously shoddy and don't r…

Another big influence is what problems can attract funding.

Re: Chart of the Decade: Why You Shouldn’t Trust Every Scientific Study You See

#102

Earlier quoted context omitted.

It's not a counterproductive limitation. In fact it should lead to more research: In this type of clinical experiment, if your data happens to show a significant result that wasn't in the pre-planned outcomes, the proper process to follow is to formulate a second course of research designed around validating that new hypothesis.

That's a nice theory. I worry that in reality a lot of those second research projects will never happen.

You're right, it may not have had that result. But I think if the pharmaceutical companies had potentially significant results there would be a strong financial incentive to follow up with a another targeted study. It would be interesting to test, look and see whether the average number of trials per unit of time, probably year, increase after this change was made. I'm curious, but not enough to actually do the work :) Always more interesting question out there than time to follow up on them.
Post reply on HN