Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Just 11% of 53 cancer research papers were reproducible

nature.com

Post reply on HN
Print view View original

131–140 of 184 posts

Re: Just 11% of 53 cancer research papers were reproducible

#131

Post by scott_karana »

btilly wrote: post #61
wheaties wrote: post #4

This is one of the reasons I went into math and software. In math there is only black and white, provably true and patently wrong. Still, part of the reason this is so is that to show any results everything must be laid on the table. CS and Math papers are therefore open by design and the world has benefited. I hope the rest of science will move in this direction.

I'm guessing you have not read Proofs and Refutations by Imre Lakatos. It is worthwhile. Also if you read on the history of mathematics, the various back and forth foundational issues raised by infinitesmals, Fourier series, proof by contradiction and the like are very interesting. Sure, it is possible for a modern mathematician to look at all that and say, They just didn't know how to do it, HERE you go. But over de…

At least those historical mathematicians acknowledged their gaps of knowledge. Cancer research is a far cry different.

Re: Just 11% of 53 cancer research papers were reproducible

#132

Post by CatMtKing »

Does reproducible in this context mean:

The paper did not document their steps sufficiently, so the authors were unable to reproduce the experiment.

-- or --

The paper relied on circumstantial evidence that was not well-understood enough to be reproduced.

-- or --

The authors were able to perform the experiment as documented in the paper, but were unable to reproduce the results.

or something else?

Re: Just 11% of 53 cancer research papers were reproducible

#133

Post by btilly »

scott_karana wrote: post #131
btilly wrote: post #61

Earlier quoted context omitted.

I'm guessing you have not read Proofs and Refutations by Imre Lakatos. It is worthwhile. Also if you read on the history of mathematics, the various back and forth foundational issues raised by infinitesmals, Fourier series, proof by contradiction and the like are very interesting. Sure, it is possible for a modern mathematician to look at all that and say, They just didn't know how to do it, HERE you go. But over de…

At least those historical mathematicians acknowledged their gaps of knowledge. Cancer research is a far cry different.

You clearly are unfamiliar with the history.

If you can find a copy, George Berkeley's The Analyst was a cogent criticism of the foundations of calculus in his day. (He didn't have answers, but he identified real problems.) A variety of books were written by mathematicians in response. Uniformly these were much lower quality, and consisted of defenses of the foundations of mathematics, rather than acknowledgements that there were real issues. His criticisms were not taken seriously until that mathematical framework completely fell apart when Fourier series constructed "obviously impossible" things.

Today we are used to a very general notion of a "function". But historically a square wave simply wasn't a function. (Because if it was, how did you know how it interacted with infinitesmals?) And getting one out of the sum of a bunch of well behaved sins, with some simple integration, was a huge shock.

Re: Just 11% of 53 cancer research papers were reproducible

#134

Post by joe_the_user »

jballanc wrote: post #67

I came to the biological sciences trained as a chemist. As I began working on my Ph.D. and variously encountered papers on cell signaling research (the field that much "cancer research" would fall into) it was blindingly apparent to me, perhaps because of my chemistry background, what was going on... Cell Signaling (and much of Biology) research is in its Alchemy phase. Now, alchemy often gets a bad rap as being comp…

This is a pretty dubious justification. Whether or not Alchemy lead to later easily reproducible results, the alchemists couldn't have engaged in an effort to present themselves as what we now know as scientists.

Oppositely, it seems logical to expect that a parade of fake and often impressive results would act as a damper to real, modest gains getting attention and funding.

Re: Just 11% of 53 cancer research papers were reproducible

#135

Post by aidenn0 »

ars wrote: post #123
aidenn0 wrote: post #108

Earlier quoted context omitted.

If you want to maximize your chances of reproducing the results, you will use identical apparatus and methods to what the original study used. That is often not possible to do just from reading the paper.

If the identical apparatus is essential to reproducing the result, then the description of that apparatus is part of the experiment. Omitting that means it's not a proper paper.

Well then this article would be testing how well the papers were written, not whether or not it was possible to reproduce the results.

Re: Just 11% of 53 cancer research papers were reproducible

#137

Post by tolmasky »

Avshalom wrote: post #125
spikels wrote: post #109

Earlier quoted context omitted.

If its not reproducible it is not science.

Huge swaths of astronomy are functionally unreproducible. We can argue about the math but many phenomena exist as a single example and/or that's basically static on our time scales. The best we can do is see if the math seems to produce similar looking structures when (sparsely) simulated.

Would there be a problem with just calling it something else other than science then? I don't see the need to bend definitions of words to account for inconvenient circumstances. I am a programmer and do hard stuff, I don't require people to call me a scientist. Mathematicians do hard stuff, they don't complain that they're not called scientists. Engineers, too, are not scientists. It's not pejorative, just a statement of fact. If what you say is correct, then what is the issue with just saying astronomy is not a scientific field?

Re: Just 11% of 53 cancer research papers were reproducible

#138

Post by enraged_camel »

jacoblyles wrote: post #104
zeteo wrote: post #93

Earlier quoted context omitted.

>Cell Signaling will, eventually, have its Joseph Priestley, its Dmitri Mendeleev. That's very optimistic, but not necessarily true (even ignoring the question of when ). Priestley succeeded so well because he developed simple, reproducible experiments (expose mercury oxide to sunlight to produce oxygen etc.) that he was happy to share for no financial gain (even traveling to others' labs to help them reproduce his w…

I don't understand why science should be a closed guild in the first place.

Because these days it is institutionalized, and in our society that means it is funded by capitalistic entities, i.e. corporations.

In other words, the profit motive dictates that making money to enrich a few individuals is more noble than sharing knowledge for the good of society.

Re: Just 11% of 53 cancer research papers were reproducible

#139

Post by Avshalom »

tolmasky wrote: post #137
Avshalom wrote: post #125

Earlier quoted context omitted.

Huge swaths of astronomy are functionally unreproducible. We can argue about the math but many phenomena exist as a single example and/or that's basically static on our time scales. The best we can do is see if the math seems to produce similar looking structures when (sparsely) simulated.

Would there be a problem with just calling it something else other than science then? I don't see the need to bend definitions of words to account for inconvenient circumstances. I am a programmer and do hard stuff, I don't require people to call me a scientist. Mathematicians do hard stuff, they don't complain that they're not called scientists. Engineers, too, are not scientists. It's not pejorative, just a stateme…

Well attempting to exclude astronomy/astrophysics from the umbrella of science would be bending the definition far more than the current status quo.

Part of this is that science isn't about experimentation, it's about observation. We perform experiments when possible so that we have more stuff to observe, or more controlled events.

Much of astronomy, atmospheric physics, geology, medicine, the "soft" sciences and I'm sure plenty of other "hard" fields are at the mercy of certain phenomena having sweet FA for data points. And I'm sure they all do what astrophysicists do: make sure that what we do have plenty of data for works; make our extrapolations with as few assumptions/rounding errors as we can; and revisit existing models anytime we find a new data point.

It's as scientific as anything it's simply going to take longer to sort out in some cases.

Re: Just 11% of 53 cancer research papers were reproducible

#140

Post by tensor »

spikels wrote: post #109
epistasis wrote: post #73

Earlier quoted context omitted.

You're assuming that only "reproducible" results are valid, which is incorrect. Results that are reproducible within a lab, but not when attempted by other people in different settings indicate that there's an unaccounted for variable, and that falls into the 89%. There's still something valuable there, it's just that we don't yet know all the variables. We could just give up, or we could try to continue study of som…

If its not reproducible it is not science.

That's not true. Part of science is doing experimentation. If an experiment doesn't reproduce, you need to find out why that is. The hypothesize and test part of the scientific process is every bit as much science as the rest, even if your tests show that an idea is wrong, or that more is going on then you thought.
Post reply on HN