Just 11% of 53 cancer research papers were reproducible • Hacker News Board

Re: Just 11% of 53 cancer research papers were reproducible

#171

Post by spikels » Fri, May 10, 2013, 7:58 AM UTC

omnisci wrote: post #129

spikels wrote: post #113

Earlier quoted context omitted.

Why do you say funding is getting cut? NIH's budget has almost doubled in the last decade [1] and many of the other funders have seen similar growth as well as new funders appearing every year. I don't think the problem is lack of funding but screwed up incentives. When medical reaseach became focused on funding the quality of the results suffered. And if the vast majority of landmark cancer research can't be reprodu…

http://news.sciencemag.org/scienceinsider/2013/05/nih-detail... Already funded grants are getting cut ~20% across the board. There is a ton of cuts going on right now to the NIH budget, google it and take a look. This has been happening for years now, trying to get an RO1 (large research grant) is becoming more and more difficult and it isn't helped by the constant changing in requirements. Every other point I agree…

You are right about the sequester cuts. I was looking at the annual numbers of the NIH front page which didn't include 2013. I wonder why 5.5% overall cuts translate to 20% cuts. The SciMag article makes it seems like they were only cutting the number of grants not the size which kinda makes sense. Perhaps they are treating funded grants worse which seems crazy. Wouldn't this potentially waste the money already spent if the project can't be finished on 20% less?

Good luck with omnisci.org this is the sort of thing that would help: open sharing of data, techniques and negative results. If this was the norm things could be very different. But one thing I have learned is it is very hard to change and organizations culture.

Re: Just 11% of 53 cancer research papers were reproducible

#172

Post by dkural » Fri, May 10, 2013, 8:06 AM UTC

I found a company to precisely fix this problem. Every dataset, every analysis, every computation fully transparent, reproducible, accessible. You can even embed it in your blog. For instance check out https://igor.sbgenomics.com/lab/public/pipelines/ if someone uses one of those, and share their task, there is full transparency + instant replication.

Re: Just 11% of 53 cancer research papers were reproducible

#173

Post by dkural » Fri, May 10, 2013, 8:11 AM UTC

codex wrote: post #154

This can be explained without bringing in accusations of fraud or incompetence [1]. Basically, if you have a bazillion experiments running and only publish the results which are significant, you still have a huge predictability problem, because most of those statistically significant results will be due to chance alone. For example, assume that only 1% of cancer experiments produce something valid and interesting, an…

this basic fact is understood by scientists and children alike. it is still either incompetence (lack of middle school statistics) or fraud (ignoring the above fact).

Re: Just 11% of 53 cancer research papers were reproducible

#174

Post by zzkt » Fri, May 10, 2013, 8:28 AM UTC

kirk21 wrote: post #78

tokenadult wrote: post #57

This editorial commentary and the article on which it is based are part of an ongoing effort to improve the quality of scientific publication in a number of disciplines. The Retraction Watch group blog http://retractionwatch.wordpress.com/ by two experienced science journalists picks up many--but not all--of the cases of peer-reviewed research papers being retracted later from science journals. Psychology as a discip…

The pressure on academics is indeed an issue. They have to teach, guide PhD's, research and publish (without mentioning controlling internal affairs). On a side note. Has anyone found a good alternative for Mendeley? I heard http://bohr.launchrock.com/ is working on sth cool.

Have you looked at http://www.zotero.org ? it covers most of the same ground as Mendeley, browser integration is quite good and the standalone app is as good or better than Mendeley

Re: Just 11% of 53 cancer research papers were reproducible

#175

Post by InclinedPlane » Fri, May 10, 2013, 9:59 AM UTC

jballanc wrote: post #67

I came to the biological sciences trained as a chemist. As I began working on my Ph.D. and variously encountered papers on cell signaling research (the field that much "cancer research" would fall into) it was blindingly apparent to me, perhaps because of my chemistry background, what was going on... Cell Signaling (and much of Biology) research is in its Alchemy phase. Now, alchemy often gets a bad rap as being comp…

I keep coming back to this essay called "Can a Biologist Fix a Radio?": http://protein.bio.msu.ru/biokhimiya/contents/v69/pdf/bcm_14...

The thing that keeps coming to mind is that it's easy to hand-wave pseudo-understanding of systems using enough jargon and vague diagrams. And if you use enough jargon and cloak things behind complex methodologies (the mechanisms to even begin investigating how cellular machinery works are enormously complex) it's easy to hide a lack of understanding. And I can't help think a similar sort of problem exists in software. The tools and terminology we have for explaining and understanding software systems leaves a lot to be desired, and one of the ways that is revealed is in the diagrams we use for describing systems. At a fairly fine grain we can use UML diagrams, but these tell us almost nothing and in complex systems simply become a giant hairball. Typically we end up describing a system as a series of layers, or as an interconnected set of services, but I can't shake the notion that such diagrams are too much like Figure 3a in the essay above than like 3b, they are the crudest sketch of a system, they don't touch on the nature or interrelation of the components.

Re: Just 11% of 53 cancer research papers were reproducible

#176

Post by viggity » Fri, May 10, 2013, 1:12 PM UTC

kailuowang wrote: post #95

vixen99 wrote: post #82

Earlier quoted context omitted.

Without qualification, I'm afraid what you're saying is parlously close to mere verbiage. Even the most ardent climate sceptic does not object to solar, wind or water energy under certain conditions. What is 'Going Green' then? In the UK it currently means, for instance, paying wind farms a million pounds sterling a day to produce no energy at all. Meanwhile poor people die because they can't afford to keep themselve…

"Going Green" does not equal to switch to renewable energy at the cost of poor people's lives for gods sake. It simply means that recognizing the global warming and have a plan to switch to renewable energy, for example, take some of the hundreds of billions of dollars of the oil companies' profits to invest on renewable energy.

proposed cap and tax carbon regimes disagree.

Re: Just 11% of 53 cancer research papers were reproducible

#177

Post by SEMW » Fri, May 10, 2013, 2:03 PM UTC

yaok wrote: post #170

anologwintermut wrote: post #166

Earlier quoted context omitted.

Science is funded entirely by corporations? Tell me, where can I buy stock in the NSF, DARPA, NIH? Or where can I find the articles of incorporation for the MacArthur Foundation or any of the other non-profits that also fund research. Or which corporation is paying my PhD stipend --- I really ought to short their stock since they were dumb enough to hire me.

Those are government entities, and the government's parent company is the corporatocracy that owns all the politicians and media. Science is a business in the service of global capitalism. Truth hurts. I will give you the MacArthur foundation and certain other non-profits. But most non-profits are also operated by the corporatocratic elite, like everything else on this planet.

You're right - if you define 'funded by corporations' sufficiently loosely (to include everything from governmental bodies to universities to charitable foundations) that the statement 'science is entirely funded by corporations' becomes a tautology, it becomes clear that science is entirely funded by corporations! I for one am shocked and appalled.

Re: Just 11% of 53 cancer research papers were reproducible

#178

Post by trentlott » Fri, May 10, 2013, 3:01 PM UTC

kanzure wrote: post #169

anologwintermut wrote: post #167

Earlier quoted context omitted.

What do paywalls possible have to do with the reproducibility of results? Anyone who has access to a lab capable of even attempting to reproduce such research is either 1) at an institution that has access to those journals or 2) can use google to find the papers anyway. Hopefully both. The fact that tax-payer funded research is put behind paywalls is a travesty , but it seems wrong to claim it actually hinders resea…

> What do paywalls possible have to do with the reproducibility > of results? Anyone who has access to a lab Actually, there's no institution subscribed to every journal. There are, in fact, many papers inaccessible to even the most funded labs. Spotty coverage.

I'm published in a highly-ranked journal that our university doesn't have access to.

Sorta funny.

Re: Just 11% of 53 cancer research papers were reproducible

#179

Post by anologwintermut » Fri, May 10, 2013, 4:23 PM UTC

kanzure wrote: post #169

anologwintermut wrote: post #167

Earlier quoted context omitted.

What do paywalls possible have to do with the reproducibility of results? Anyone who has access to a lab capable of even attempting to reproduce such research is either 1) at an institution that has access to those journals or 2) can use google to find the papers anyway. Hopefully both. The fact that tax-payer funded research is put behind paywalls is a travesty , but it seems wrong to claim it actually hinders resea…

> What do paywalls possible have to do with the reproducibility > of results? Anyone who has access to a lab Actually, there's no institution subscribed to every journal. There are, in fact, many papers inaccessible to even the most funded labs. Spotty coverage.

No, no institution subscribes to every journal. However, see point 2. Given the title and author, how many of those paper's can't you find with google / google scholar(or emailing the author).

Paywalls for publicly funded research our wrong, but the cases when it prevents people at research institutions from doing research seem to be tiny.

Re: Just 11% of 53 cancer research papers were reproducible

#180

Post by anologwintermut » Fri, May 10, 2013, 4:28 PM UTC

return0 wrote: post #168

anologwintermut wrote: post #167

Earlier quoted context omitted.

What do paywalls possible have to do with the reproducibility of results? Anyone who has access to a lab capable of even attempting to reproduce such research is either 1) at an institution that has access to those journals or 2) can use google to find the papers anyway. Hopefully both. The fact that tax-payer funded research is put behind paywalls is a travesty , but it seems wrong to claim it actually hinders resea…

If they were open, you would have an online service that could automatically point contradicting studies, and even make suggestions.

Even if they are closed, you can link to the papers (unless the DMCA metastasized again). Granted, someone can't make an automatic tool to find such studies, but that would be hard anyway and presumably the authors are aware of the study they are refuting and could tag them.

Now, people trying to make policy decisions could not read or evaluate the studies. If we assume they are capable of doing so in the first place( i.e. their not Lamar smith), then we have an actual argument for getting rid of paywalls. I still think the best one is its a waste of money and a tax on universities/ grants.