Live data from Hacker News

Just 11% of 53 cancer research papers were reproducible

nature.com

41–50 of 184 posts

Re: Just 11% of 53 cancer research papers were reproducible

#41
post #24

After decades of publish-or-perish sweatshop science I'm sure the great shining archive of scientific truth is sort of like an inbox with no spam filter.

Such a powerful sentence.

Lately science looks like olympic games without anti-doping control...

Re: Just 11% of 53 cancer research papers were reproducible

#42
post #30

Earlier quoted context omitted.

> "those authors required them to sign an agreement that they would not disclose their findings about specific papers" I don't know why this doesn't cause a huge stir in the scientific community. Seems like everyone is fine with people sweeping their negative data under the rug. Shameful is a very mild word for that.

Agreed, but this study was specifically designed to highlight the problem not solve it. And the results would be useless, or non-existent, if they restricted themselves to scientists that would co-operate fully with the process and the possible "shaming" afterward.

I think it also highlights the underlying mentality that prestige matters more than integrity and "if everyone does it then it's fine".

If tax-funded scientists can't uphold high standards for themselves, the public is entitled to demand them to do so.

Re: Just 11% of 53 cancer research papers were reproducible

#43
post #7

After hearing similar things about psychology papers, this is rather disconcerting. This is why I am a climate change skeptic, I don't know whether it is happening due to CO2 or not, but I am confident that science can't be confident when they can't do control experiments. You can't control for any variable when it comes to the climate, let alone all the reasonable ones. We have infinitely more capabilities to contro…

You took an almost entirely unrelated story and made it about your political hobbyhorse. Please keep politics off HN.

The article was not about cancer. The whole point of the discussion is about reproducibility in scientific research. I was merely applying the same discussion and concept to a different branch of science. HN is a place for nerds to discuss interesting things. Things like science and it's applications. As far as I'm concerned, that is what I did.

Re: Just 11% of 53 cancer research papers were reproducible

#44
post #35

Pediatric Cancer gets only 4% of Federal funding and less than 1% of Private Drug company research. Also American Cancer Society less than 5 cents on the dollar goes to research. Pediatric Cancer is the leading cause of death for 15 and under. So to see this funding for cancer get thrown away by fraud is extremely upsetting.

Those are some pretty misleading statistics.

First off, pediatric cancer is NOT the leading cause of death for those 15 and under[1]. It's up there, but "unintentional injury" takes twice as many lives.

Second of all, not many children die (1200 per year), so even if something is the leading cause of death, it doesn't mean it happens very often. Cancer kills 10 times as many people between the ages of 35-44. If you add up all adult cancers it's probably 20-30x the number of deaths.

Also, just because money goes to adult cancer, doesn't mean it can't help. Most drug are developed for adults first and then tested in children. It's not like people are ignoring pediatric cancer.

Before you get so upset you should really sit down and think these things through.

[1]http://www.cdc.gov/injury/wisqars/pdf/10LCID_All_Deaths_By_A...

Re: Just 11% of 53 cancer research papers were reproducible

#45
post #27

I have been in discussions about this with one of my friends working in academic materials research. Its amazing the amount of work today done by scientist at universities writing code without very basic software development tools. I'm talking opening their code in notepad, 'versioning' files by sending around zip files with numbers manually added to the end of the file name, etc. This doesn't even begin to scratch t…

This is an entirely different issue than code; code mostly does the same thing when you run it twice. There's no such guarantee in biology. A cancer cell line growing in one lab may behave differently than descendants of those cells in a different lab. This may be due to slight differences in the timings between feeding the cells and the experiments, stochastic responses built into the biology, slight variations between batches of input materials for the cells, mutations in the genomes as the cell line grows, or even mistaking one cell line for another.

Reproducibility of software is a truly trivial problem in comparison.

Re: Just 11% of 53 cancer research papers were reproducible

#46
post #23

Wow, that's sad. Are we really so blind when it comes to cancer research? I recall that oncology journals usually have a ludicrously high impact index (as ludicrously as 5 digit IIRC); that means there are a lot of citations, which is an indicator that there is a lot of research going on. And with a lot of research going on, well, you can expect a lot of false positives. So, I'm wondering, could this be a case of che…

"And with a lot of research going on, well, you can expect a lot of false positives."

Why would you expect a lot of false positives? Aren't results supposed to be reproducible at least 95% of the time?

Re: Just 11% of 53 cancer research papers were reproducible

#47
post #23

Wow, that's sad. Are we really so blind when it comes to cancer research? I recall that oncology journals usually have a ludicrously high impact index (as ludicrously as 5 digit IIRC); that means there are a lot of citations, which is an indicator that there is a lot of research going on. And with a lot of research going on, well, you can expect a lot of false positives. So, I'm wondering, could this be a case of che…

[deleted]

Re: Just 11% of 53 cancer research papers were reproducible

#48
post #23

Wow, that's sad. Are we really so blind when it comes to cancer research? I recall that oncology journals usually have a ludicrously high impact index (as ludicrously as 5 digit IIRC); that means there are a lot of citations, which is an indicator that there is a lot of research going on. And with a lot of research going on, well, you can expect a lot of false positives. So, I'm wondering, could this be a case of che…

But who is going to pay to reproduce that research? What if that research took years to do?

Re: Just 11% of 53 cancer research papers were reproducible

#49
What is more troubling is not that so few of these results results are reproducible, but that it appears almost no-one is trying to reproduce the results of earlier studies. The ability of the scientists who wrote the paper to even get access to the resources necessary to try and reproduce the results is limited.

I'm reminded of Feymann's Cargo Cult Science:

"I was shocked to hear of an experiment done at the big accelerator at the National Accelerator Laboratory, where a person used deuterium. In order to compare his heavy hydrogen results to what might happen with light hydrogen he had to use data from someone else's experiment on light hydrogen, which was done on different apparatus. When asked why, he said it was because he couldn't get time on the program (because there's so little time and it's such expensive apparatus) to do the experiment with light hydrogen on this apparatus because there wouldn't be any new result. And so the men in charge of programs at NAL are so anxious for new results, in order to get more money to keep the thing going for public relations purposes, they are destroying--possibly--the value of the experiments themselves, which is the whole purpose of the thing. It is often hard for the experimenters there to complete their work as their scientific integrity demands."

For anyone who hasn't read it, the whole thing is excellent http://www.lhup.edu/~DSIMANEK/cargocul.htm

Re: Just 11% of 53 cancer research papers were reproducible

#50
post #44
post #35

Pediatric Cancer gets only 4% of Federal funding and less than 1% of Private Drug company research. Also American Cancer Society less than 5 cents on the dollar goes to research. Pediatric Cancer is the leading cause of death for 15 and under. So to see this funding for cancer get thrown away by fraud is extremely upsetting.

Those are some pretty misleading statistics. First off, pediatric cancer is NOT the leading cause of death for those 15 and under[1]. It's up there, but "unintentional injury" takes twice as many lives. Second of all, not many children die (1200 per year), so even if something is the leading cause of death, it doesn't mean it happens very often. Cancer kills 10 times as many people between the ages of 35-44. If you a…

"Most drug are developed for adults first and then tested in children."

It was also be unethical to develop drugs for children without first testing them on adults, since children can't give informed consent.

Post reply on HN