Cargo cult science (1974)

> 1) Not use the placebo, because even though it seems to work for you, it's been scientifically shown in trials to do nothing

This is the opposite of a placebo. A placebo is a treatment that has been scientifically shown in trials to do something even though it shouldn't. For example, taking a pill or drinking a cup of "medicine" might make your headache feel better even though the pill is just sugar and the medicine is just colored water, and just eating a spoonful of sugar or drinking the same amount of water doesn't have the same effect if you don't believe it's medicine.

We use placebos as the comparison in clinical trials precisely because placebos do something, and we want to show that taking the pill with the actual medicine does more than just taking a sugar pill does. Because of it doesn't--might as well just take the sugar pill.

Placebos do something, and if that something is beneficial then by all means take them. If I can take a sugar pill that makes me feel better I don't really care whether it's psychosomatic or pharmacological. The FDA cares, because they want to encourage real treatments and discourage quackery.

But it is not only people cargo culting at home. It is also people doing that on the job. Instead of looking at various alternatives and truly evaluating them, looking at benefit and downsides each brings, people jump to whatever is the current hype or what is fashionable and then justify it with a circular argument of popularity.

Instead of immediately when the talk is about clusters jumping to kubernetes, people should think about whether the conceptual overhead is actually worth it. They should think about, whether the Google scale tool is truly appropriate for what they want to do. Similar situation of course with all the hyped JS frameworks, when a simple statically rendered website/page would do.

Basically it is not engineering any longer, but cargo cult and hype. And that is how shitty products are made.

I don't see it as a general public problem but I actually do see it as an ongoing scientific training problem. Biology PhD programs these days spend a lot more time teaching prior biology results than they do having students engage with what it means to be a scientist, in the limited time that they teach at all.

"Students" are then sorted into a lab where they learn the specific scientific cookbook of the subfield and in some cases never even try to apply it truly on their own before graduating, because they are used as a lab monkey for 4+ years. Collaborative side projects or substantial additional course work are rare, so students are often functionally the same as a lab employee.

The lab has minimal external incentive to train the person to become a good scientist, and there isn't much cultural pressure either, so the training focus is instead to become a good [hyperspecific topic] lab member.

To graduate a student needs a deep understanding of what was previously shown in their niche, but they really do not need to engage with much at all to do with meta science, and it is wholly possible to graduate without ever demonstrating the ability to ask good scientific questions from the genesis of a project through to it's completion, even in the context of the norms of said niche subfield.

Student experiences of course vary widely, and even for those that do the menial slog version of a bio PhD, the attitude surrounding it varies. Some seem to be happy for the guidance when their advisor is like this while others resent the lack of intellectual freedom.

Regardless, I find this to be pervasive in the environments I've been able to observe, even at a few of the supposedly top life sciences PhD programs in the US. To think deeply about meta science in any official capacity as part of PhD training requires you to go out of your way.

Meanwhile on the other end of the spectrum it is possible to receive a "top" PhD and stay in academia for quite a long time through only a combination of hard work and learning the experimental best practices of the field as if it were a trade (and that might be underrating the nuance in the trades honestly, I don't know enough to say).

I do think it is a difficult problem to avoid to at least some extent, because open questions in biology increase in complexity quite a lot over time, with a big time and financial commitment required to do a number of projects these days. Without radical changes in labor sources for wet lab work I'm not sure the issue can be sufficiently addressed.

I also imagine that even in less expensive/slow fields there is an increasingly unapproachable body of prior knowledge to learn, and people don't want to be trainees forever. So I believe a large philosophical paradigm shift might be needed too.

I'd love to see multiple different types of PhDs introduced with different graduation requirements, funding structures, and intended postgrad career paths, while maintaining the core thesis writing experience across the board.

Ironically, scientists love to do this! Take a new area with promising results, hype the shit out of it until the worst and/or least ethical scientists truly create a pseudoscientific tangent out of it, and then drop the area entirely for a decade because now anything vaguely associated with it is taboo. Labeling real science as cargo cult science sounds to me like cargo cult science with extra steps.

Granted such a violent burst doesn't always happen when an academic bubble is created. But at times I've found academic backlash to be shockingly irrational in its breadth. I think the most common factor is when the pseudoscience makes it to the point of having an impact on actual humans. But it is still sad to see a state when even tenured faculty will shy away from public discussion of certain questions because of vague associations, as it feels entirely unscientific to me.

The funniest part is when the area reemerges with a new name, as if extremely surface level branding is actually important to receiving grants or publishing papers. Well I suppose it is unfortunately, but this is one of those cases that feels so thinly veiled it might as well be scientific satire.

My personal favorite example is the galvanic skin response hype from 50+ years ago, which morphed into more of a fringe science primarily because of flaws with lie detection applications - flaws that should've been extremely obvious concerns to test just based on the OG literature + common scientific sense. In the last decade or so it's been revamped as electrodermal activity and unsurprisingly shown great promise for certain health applications.

EDA is presently the main focus of a big group at MIT, it has been included in some Fitbit models, it is used in FDA approved devices for epilepsy monitoring, and overall the relevant publications per year keep trending back up. A far cry from the looks you'd get if you discussed galvanic skin response in certain supposedly scientific circles back in the day.

In absolute numbers the rebranded GSR has actually already fully recovered from the couple decades of low research activity, though relative to total academic output now versus then it probably still has some distance to go.

I love that observation.

Science without transparency is just politics.