The Irreproducibility Crisis of Modern Science

My objection is to conflating them with scientific matters, i.e. empirical claims, especially about human biology. I worry that doing so actually weakens our arguments against such evil. We shouldn't build into our case against them scientific claims on which we may ourselves be proven wrong. It's not necessary.

(And, much as I enjoy using words like diversity in their older meanings, in the hope of tripping people into thinking, here I resisted the temptation. Intellectual monoculture was however a feature of most of the evil regimes of recent times, too, and one of their weaknesses.)

However, this paper isn't a study. It's just an essay. And while, yes, the paper in question does highlight numerous issues in reasonable and rational ways, it also uses this as the breakout quote on page 36:

> Scientists readily accept results that confirm liberal political arguments, and frequently reject contrary results out of hand.

Now, there's a couple problems with this statement. The most obvious is that it implies blame on liberal politics. On the same page they detail that the percentage of self-identified liberal psychologists at a conference (~800 of 1,000) wildly outnumbers the number of self-identified conservative psychologists at the same conference (3 of 1,000). Well, this conference is the Society for Personality and Social Psychology, which is based in southern California. Would that bias attendance? However, even if we assume that distribution is correct, the quoted statement doesn't say anything about the conservative scientists. Would it also be correct that conservative scientists readily accept results that confirm their political arguments? We don't know. Maybe it would be more correct to say this:

> Scientists readily accept results that confirm pre-existing personal political arguments, and frequently reject contrary results out of hand.

Instead, this paper chooses to imply that scientists who are more liberal are somehow inherently more biased. If we assert that good scientists are those who don't allow political arguments to influence their acceptance of their results, then the natural conclusion to draw from that is that being liberal is incompatible with being a good scientist.

Don't get me wrong, this paper makes a lot of legitimately good points with good examples, but the whole section on groupthink comes across a little bit biased itself. However, it's existence means I can't shake the feeling that maybe I'm missing something.

So having show that there's a reproducibility problem, the paper concludes. Then very end of the paper, the afterword lists these points introduced seemingly out of the blue:

> Symptoms of Pathological Science:

> 1. The maximum effect that is observed is produced by a causative agent of barely detectable intensity, and the magnitude of the effect is substantially independent of the intensity of the cause.

> 2. The effect is of a magnitude that remains close to the limit of detectability; or, many measurements are necessary because of the very low statistical significance of the results.

> 3. Claims of great accuracy.

> 4. Fantastic theories contrary to experience.

> 5. Criticisms are met by ad hoc excuses thought up on the spur of the moment.

> 6. Ratio of supporters to critics rises up to somewhere near 50% and then falls gradually to oblivion.

These are from Irving Langmuir (https://en.wikipedia.org/wiki/Irving_Langmuir) and his criticisms on research conducted with unconscious biases or subjective elements. He meant it as a criticism of research like cold fusion.

Now, there's not any real one-for-one discussion of these items or how they relate to what came before. They're just presented basically as-is as the final portion of the essay with few notes that they correlate to what came before. You'll notice, however, they they all call into question the credibility of the scientist as well as the credibility of the scientific community, and not the quality of their research or data.

However, let's say you're politically conservative, and you're reading this essay. Maybe you don't believe in climate change. You get to the end of this paper, and it tells you that in spite of the claims of accuracy of the data, in spite of the consensus of the community, in spite of all the evidence, if it's contrary to [your] experience or if the defenses to criticism feel invented on the spot or if you keep hearing claims that support is waning, and apparently especially if it's liberal, then you've found a legitimate case of bad science.

I don't think that's necessarily the point they're trying to make here, but I really don't like the way this afterword is presented. It's introducing new concepts and arguments and then handwaves away having to justify them or explain their relevance.

This is a good example of why it's not so simple as you're making out. Earlier, you put biology on your list of unreliable branches of science, and you claimed that you could know that the racial science of the 30s wasn't "real science" simply because it consisted of "studies about people". But vaccine science falls into both those categories!

In reality you're deciding whether or not any given branch of science is methodologically sound not by examining any actual studies in the relevant disciplines, but simply on the assumption that any study which has arrived at the "wrong" conclusion must have had a bad methodology.

By way of analogy, when someone tells you he believes global warming is impossible because it would go against the Bible, you don't have to wonder that much what he would do about carbon emissions.

Of course it can. In fact, it can tell you exactly how much more likely one case is versus the other. P=0.05 means that there is a 5% chance that the null is true and hence a 95% chance the the null is false.

> says something about the likelihood of the _hypothesis_ (like you're trying to do)

No. I am simply saying that there is a lower bound on the rate of false positives caused by the choice of the threshold p value. If you chose P<=0.05 then you will have an error rate of at least 5%. The actual error rate is almost certainly higher (that's what "lower bound" means) for a host of other reasons.