Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Most scientists 'can't replicate studies by their peers'

bbc.co.uk

Post reply on HN
Print view View original

351–356 of 356 posts

Re: Most scientists 'can't replicate studies by their peers'

#351

Post by FabHK »

jszymborski wrote: post #301
Balgair wrote: post #153

Earlier quoted context omitted.

A lot of the commenters are trying to come up with technical solutions to the issue. It belies their incomplete study of the 'softer' sciences and the difficulty of biological studies. Tech solutions won't work, bureaucratic solutions wont work, more data wont work. Guys, the bio side of things is incredibly complicated and trying to set controls that are achievable in your time and budget is the heart of these field…

While we're sharing confusing pathways, here's a visualisation of _some_ of the known metabolic pathways: http://biochemical-pathways.com/#/map/1

It is awesome to get a visual, visceral reminder occasionally about how much we know these days.

Re: Most scientists 'can't replicate studies by their peers'

#352

Post by FabHK »

hyperion2010 wrote: post #183
nacc wrote: post #28

"all you have to do is read the methods section in the paper and follow the instructions." I wish science was that simple. The methods section only contains variables the authors think worth controlling, and in reality you never know, and the authors never know. Secondly, I wish people say: "I replicated the methods and got a solid negative result" instead of "I can't replicate this experiment". Because most of the t…

My PhD work is trying to address this by developing better ways for scientists to record and communicate their methods/protocols. Methods sections are NOT about providing the information needed to reproduce the work, often even the supplemental information is insufficient. They are a best guess at what needs to be done, and the methods section is often massively condensed due to editor demands and cites other papers…

Have you looked at Common Workflow Language (CWL), see below?

Friend of mine has been experimenting with wrapping it all up in Docker containers! :-)

> a specification for describing analysis workflows and tools in a way that makes them portable and scalable across a variety of software and hardware environments, from workstations to cluster, cloud, and high performance computing (HPC) environments. CWL is designed to meet the needs of data-intensive science, such as Bioinformatics, Medical Imaging, Astronomy, Physics, and Chemistry.

http://www.commonwl.org

Re: Most scientists 'can't replicate studies by their peers'

#353

Post by FabHK »

benrawk wrote: post #38

I am a social scientist studying human behavior, and this is a huge problem in the field. Myself and my statistician friends who analyze the literature have basically concluded that most extremely "novel" and "surprising" findings in the literature aren't even worth trying to replicate (remember, replications cost money to run, so before you start you have to make some judgment about the likelihood of success.) This…

Rolf Degen runs a great and responsive Twitter on results and replication failures in social sciences, not sure he has a blog.

https://twitter.com/DegenRolf

http://retractionwatch.com/2016/08/05/meet-one-of-science-pu...

Re: Most scientists 'can't replicate studies by their peers'

#354

Post by _Codemonkeyism »

_Codemonkeyism wrote: post #223

Simple. You can only publish to Nature etc. if your results are replicated twice by others and you open up all your raw data. Short time later only real science will be published. The amount of published results will massively go down and Nature and other will not makes as much money. Now you know the reason why publishers publish results that can't be replicated. Or listen to anything from Feynman about real science…

This comment got voted down.

Days later Nature posts "No publication without confirmation" http://www.nature.com/news/no-publication-without-confirmati...

Re: Most scientists 'can't replicate studies by their peers'

#355

Post by mdpopescu »

dekhn wrote: post #148
mdpopescu wrote: post #140

Earlier quoted context omitted.

How can there be any other answer than "yes"? The point of science was that it's reproducible; you are saying that what you did was not; therefore, it wasn't science. What is the difference between your paper and an Uri Geller "experiment"? Both are extremely hard to duplicate for objective reasons so their results have to be accepted based on reputation. (Imagine someone trying to publish the same thing but using "S…

Actually, reproducibility is not considered a necessity in science. It is factually false to say that a person who carries out an experiment in good faith and shares their methods and results is not scientific- they just aren't satisfying a criterion which is a good thing to have. It's pretty clear what's different between my paper and a Uri Geller experiment. If you can't see the difference, you're either being obst…

> Actually, reproducibility is not considered a necessity in science.

Incidentally, Scott Alexander just published an article [1] with a great quote:

"Peer review is a spam filter. Replication is science."

[1] https://slatestarcodex.com/2017/02/27/ssc-journal-club-analy...

Re: Most scientists 'can't replicate studies by their peers'

#356

Post by dTal »

SilasX wrote: post #346
dTal wrote: post #338

Earlier quoted context omitted.

I was pointing out the category confusion. Science teachers don't purport to tell you what scientists actually do (history or sociology teachers might, I suppose, but they don't usually "make such a big deal" out of the scientific method). They purport to teach science. As you say - it's not a comforting lie, it's a command.

The "comforting lie" I was referring to was "this is how scientists actually do it and why you can trust the results".

Erm, yes. We established that. And I assert that this is not what science teachers teach.
Post reply on HN