The Irreproducibility Crisis of Modern Science • Hacker News Board

Re: The Irreproducibility Crisis of Modern Science

#71

Post by wRastel27 » Tue, Apr 17, 2018, 4:09 PM UTC

kiziaru wrote: post #57

HarryHirsch wrote: post #10

Earlier quoted context omitted.

It's a position paper from a conservative think tank and needs to be treated as such.

What makes it a conservative think tank?

From the intro: The new, alternative view, was that college and universities should be places where fresh ideas untrammeled by hidden connections to the established structures of power in American society should have the chance to develop themselves. In practice this meant a hearty welcome to neo-Marxism, radical feminism, historicism, post-colonialism, deconstructionism, post-modernism, liberation theology, and a host of other ideologies.

Also from the intro: The National Association of Scholars (NAS) has long been interested in the politicization of science.

Re: The Irreproducibility Crisis of Modern Science

#72

Post by genericone » Tue, Apr 17, 2018, 4:11 PM UTC

lotsofpulp wrote: post #67

merpnderp wrote: post #48

Earlier quoted context omitted.

So you'd ignore one of the most horrific and important examples of widely accepted science going wrong because of an internet meme?

Science didn't go wrong. The data was bad, the processes were bad, there was no replication. How was that science?

I think that is simply another misinterpretation of what people mean when the word science is used.

What was said above was "one of the most horrific and important examples of widely accepted science". Important words here are "widely accepted science". So science didn't go wrong, the LABEL of science was attached to something that didn't meet the scientific method. That something was then sold to people in power, and the science LABEL was used to convince people of the correctness of that something.

Re: The Irreproducibility Crisis of Modern Science

#73

Post by arafa » Tue, Apr 17, 2018, 4:15 PM UTC

What I'd like to know is if the "irreproducibility crisis" is really some combination of "sample size was too small" and "effect size was too small". When I went through a lot of these studies myself and saw the ones that don't reproduce, I saw this theme over and over. "P-hacking" is less of a concern to me when the effect is real and widespread.

It's so bad now that for any article/study, I look at the sample size first. If it's too small (especially < 100) or they don't say I just ignore it. And if you don't publish or give some estimate of your effect size I just think about it directionally but don't give it much weight mentally.

Re: The Irreproducibility Crisis of Modern Science

#74

Post by emodendroket » Tue, Apr 17, 2018, 4:20 PM UTC

lmm wrote: post #59

emodendroket wrote: post #25

Earlier quoted context omitted.

Even if science were perfect it couldn't tell us about moral axioms or value judgments. Additionally, using science and other expert judgments can easily tend toward simply choosing experts and studies that align with whatever policy you want to push for unrelated reasons, or outright seeking to manipulate research for political ends.

> Additionally, using science and other expert judgments can easily tend toward simply choosing experts and studies that align with whatever policy you want to push for unrelated reasons, or outright seeking to manipulate research for political ends. Of course it can, but that's equally true for any other way of making decisions. So it's not a downside of science in particular.

It's a downside of any way of decision-making that starts to get imbued with a sense of being beyond question or argument. Many invocations of "science" and "rationality" seem to treat those things more as oracles that "tell us" whatever it is they happen to want us to think than as processes or tools for investigation.

Re: The Irreproducibility Crisis of Modern Science

#75

Post by thedonkeycometh » Tue, Apr 17, 2018, 4:22 PM UTC

This is part and parcel of Scientific method. I'd be suspicious if all experiments were successful all the time. eyes pharma

Re: The Irreproducibility Crisis of Modern Science

#76

Post by danharaj » Tue, Apr 17, 2018, 4:23 PM UTC

Lets take a page from Marx. Science is many things, in particular a relationship between capital and labor. The scientific method is a wonderful idea, but it is subordinate to the economic forces that underlie scientific activity. Look at the conflicts and contradictions between those doing science (labor) and those deciding the science to be done (capital), and that is the ultimate source of these crises.

The executive summary lists 40 points on how to improve the reproducibility of science. A bit over half of them are addressed to the sources of capital such as private organizations, universities, and governments. I think many of those points are good. However, I don't think the other points, the ones that recommend doing science in different ways, have a good punch. Even if you fix the problems there are today, so long as science is a rat race of trying to get grant money to stay afloat while burning out grad student after grad student, I think other pathological practices will creep in as a completely rational response on the parts of scientists to a hostile ecosystem. There's just a very big gap between how science should be done and what capital owners want from science.

Re: The Irreproducibility Crisis of Modern Science

#77

Post by samfriedman » Tue, Apr 17, 2018, 4:23 PM UTC

ItsMe000001 wrote: post #18

HarryHirsch wrote: post #10

Earlier quoted context omitted.

It's a position paper from a conservative think tank and needs to be treated as such.

> and needs to be treated as such What does that even mean? You have one point "it's a conservative think tank" - which does not mean anything - coupled with another empty statement. Now I sure don't mind criticism of that entity, but I find it ironic that it is done with such an empty statement that's just a not very cleverly disguised type of ad hominem attack. May I suggest you take some time actually reading the…

It is absolutely relevant to consider the political motivations of publishers of reports like this, when the publisher is a politically active organization as opposed to an independent research group.

While the report itself may not push an obvious partisan agenda, a critical reader should consider the conservative agenda of the NAS in relation to why they might publish such a report. In the conservative wing's long-standing opposition to science, it's not far-fetched to view this report as being published tactically -- even if all of its points are well-founded -- as something to point to when future efforts attempt to undermine science-based policy on issues like climate change, stem-cell research, abortion, etc.

Especially when the President of the NAS is a public "climate skeptic".

https://www.chronicle.com/blogs/innovations/bottling-up-glob...

Re: The Irreproducibility Crisis of Modern Science

#78

Post by HarryHirsch » Tue, Apr 17, 2018, 4:23 PM UTC

ItsMe000001 wrote: post #18

HarryHirsch wrote: post #10

Earlier quoted context omitted.

It's a position paper from a conservative think tank and needs to be treated as such.

> and needs to be treated as such What does that even mean? You have one point "it's a conservative think tank" - which does not mean anything - coupled with another empty statement. Now I sure don't mind criticism of that entity, but I find it ironic that it is done with such an empty statement that's just a not very cleverly disguised type of ad hominem attack. May I suggest you take some time actually reading the…

A position paper aims to make a political position appealing, a study about the sociology of science on the other hand would seek explanations for a position. Pointing out the purpose of something isn't a character attack.

Re: The Irreproducibility Crisis of Modern Science

#79

Post by Erlich_Bachman » Tue, Apr 17, 2018, 4:23 PM UTC

lisper wrote: post #68

The site is down so I can't read the original report, but I've read reports on this topic in the past so I'm going to chime in with some "usual suspects" caveats: 1. No result is 100% reproducible because you can never completely reproduce the conditions of any experiment. The best you can hope to do is to reproduce the conditions that matter , but enumerating those has to be part of the theory you are testing, and s…

4.)

You mean that papers are published with PWhen did this become a "standard"? Most papers I have seen in respectable journals and other sources would not strive for such a high Pvalue, it would be much lower like 0.001.

Re: The Irreproducibility Crisis of Modern Science

#80

Post by 323454 » Tue, Apr 17, 2018, 4:23 PM UTC

timtadh wrote: post #62

In response to several threads here: it is important to distinguish when scientists are self critical vs. when non-scientists are critical of the scientific method. For instance, there is a long history of scientists criticizing how the scientific process is currently conducted for the purposes of improving the scientific endeavor. That work is sometimes used by non-scientists who question the overall scientific meth…

I agree with your overall point, but technically speaking it is logically valid to prove a hypothesis false by first assuming it and then deriving a contradiction, even when the contradiction is the negation of the original hypothesis (as it is in your example).

What you should have said is that some critics start with the premise "the scientific method does not arrive at truth", and then use other people's arguments that depend on the premise "the scientific method arrives at truth" to support their claim, which is indeed logically invalid.