21–30 of 184 posts
An article claiming that only 11% of findings could be reproduced doesn't cite its sources, thus rendering the findings of the article unreproducible. Nice work!
This is one of the reasons I went into math and software. In math there is only black and white, provably true and patently wrong. Still, part of the reason this is so is that to show any results everything must be laid on the table. CS and Math papers are therefore open by design and the world has benefited. I hope the rest of science will move in this direction.
I recall that oncology journals usually have a ludicrously high impact index (as ludicrously as 5 digit IIRC); that means there are a lot of citations, which is an indicator that there is a lot of research going on. And with a lot of research going on, well, you can expect a lot of false positives. So, I'm wondering, could this be a case of cherry-picking or some kind of selection bias? It wouldn't be difficult to select a lot of bogus-sounding research and test it.
An article claiming that only 11% of findings could be reproduced doesn't cite its sources, thus rendering the findings of the article unreproducible. Nice work!
Most published scientific research papers are wrong, according to a new analysis. http://www.newscientist.com/article/dn7915-most-scientific-p... Ioannidis JPA (2005) Why Most Published Research Findings Are False. PLoS Med 2(8): e124. doi:10.1371/journal.pmed.0020124 http://dx.doi.org/10.1371/journal.pmed.0020124
Is this analysis a published scientific research?
It's a good thing that people can measure these things and ring the alarm
I'm talking opening their code in notepad, 'versioning' files by sending around zip files with numbers manually added to the end of the file name, etc.
This doesn't even begin to scratch the surface of the 'reproducible results' problem. Often times, the software I've seen is 'rough' to be kind. Most times its not even possible to get the software running (missing some really specific library or some changes to a dependency which haven't been distributed) or its built for a super specific environment and makes huge assumptions on what can 'be assumed about the system.' This same software produces results which end up being published in journals.
If any of these places had money to spend, I think there could be a valuable business in teaching science types how to better manage their software. Its really unfortunate that outside of a few core libraries (numpy, etc.) the default method is for each researcher to rebuild the components they need.
I'm surprised about only 11% of results being reproducible. It seems lower then I'd expect. I agree we don't want to optimize for reproducibility, but obviously there is some problem here that needs to be addressed.
An article claiming that only 11% of findings could be reproduced doesn't cite its sources, thus rendering the findings of the article unreproducible. Nice work!
Exactly. This should be the top comment. Are we just supposed to assume their work was true based on trust? Perhaps this 'study' was actually just a fleecing meant to show that we take too many studies to be true based on simple trust.
An article claiming that only 11% of findings could be reproduced doesn't cite its sources, thus rendering the findings of the article unreproducible. Nice work!
Nature wrote an editorial justifying the situation, you can read it here http://www.nature.com/nature/journal/v485/n7396/full/485041e...
I don't know why this doesn't cause a huge stir in the scientific community. Seems like everyone is fine with people sweeping their negative data under the rug. Shameful is a very mild word for that.