Time to assume that health research is fraudulent until proven otherwise?

During a debate or conversation, many people cite studies in support of their point of view. This is a problem because now the other person is swamped with a dozen studies to analyze and debunk before proving he’s right. And academia is producing huge volumes of these bullshit studies so no matter how you slice it, a huge unnecessary burden has been created of digging through all of them and circling the flaws.

Thankfully, there is an emerging cultural mechanism to deal with this in the growing “epidemiology is bullshit” sentiment. This is good because it reduces the bulk of bullshit that will ultimately need to be processed and debunked. If the study is epidemiology just cross it out by default. Those studies need to burn in hell. Shine the light of day on them and brandish the holy crucifix whenever you see one.

The only thing worse than a science denier is a person who blindly parrots study titles without ever reading the body of the paper let alone understand it. People complain endlessly about armchair scientists who are spreading misinformation based on their uneducated assessment of scientific data. And the people who complain about this are always the same people who cite studies that they don’t understand like complete idiots, spreading misinformation just as widely.

There's one link in the chain here missing that some people here seem to be ignoring. The authors of this post (while entirely correct) draw no link between "bad data" (which is doubtlessly responsible for a large number of "bad papers"/"bad trials") and "bad clinical practice."

I don't know a single clinician who would base their care on the findings of a single-center RCT of the kind described in this article. Or the findings of a meta-analysis of single-center RCTs, for that matter.

Bad data happens in multi-center RCTs too, and in fact that's what I'm focused on, but a lot of work already (and therefore $, for the cynical) goes into the validation of data (see [1] for a brief description). Phase III clinical trials in the west practically require a robust multi-center RCT, where systemic fraud is very difficult to perform (but not impossible [2]). By the time a Phase III trial is conducted, the efficacy of the drug can already be estimated, and the focus of the drug company (who yes, often fund these trials) is to conduct a trial which is unimpeachable in the face of a regulatory board (who are generally good at their jobs, although the revolving-door tends to reduce public trust and should be legislated away).

In short, I support most of the proposed changes to incentives around publish-or-perish. I reject the notion that these incentives are (currently) significant drivers of decreased quality of standard of care in the West. I think global governance structures, as suggested in this article, could improve understanding among both clinicians who are not necessarily scientists and the general public about just how validated a given standard of care is.

tl;dr Most good evidence-based practitioners already think this way -- not because they inherently believe fraud is rampant, necessarily, but because evidence says the kinds of studies where fraud is most prevalent are untrustworthy for other reasons.

[1] doi:10.1177/1740774512447898

[2] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4340084/