How not to be a crank: rules for not being a science-dick

Gelman does real, solid work and confines his righteous takedowns to a side hobby. And his criticisms are directed at individual, specific cases, with evidence, and he reserves most of his wrath for repeat offenders rather than one-off mistakes, which are unavoidable given enough projects. He confines criticisms to his area of expertise, which is statistics.

Ioannidis' claim to fame is writing a hand-waving, philosophical argument in order to cast doubt on all research at once. Followed up shortly by a paper that implied (but didn't directly state!) that regression to the mean on replication implies error or fabrication in the original. And he has ridden this sort of tired argument into a lucrative and prestigious career and Stanford appointment. Never mind the collateral damage on funding and morale. My theory is that people pay attention to Ioannidis and wring their hands about these highly generic issues for the same reason most people pretend to be highly, highly concerned about sexual harassment/discrimination in the workplace -- they're praying that if they act concerned enough, the hammer will fall somewhere else. But indiscriminate criticism like this is highly toxic to science (or anywhere else, for that matter, and has the side effect of letting real bad actors off the hook).

A further moral of the story is that, for better or worse, I don't trust and generally ignore criticisms that do not come from someone who has invested time into a field and made useful contributions to it. 'Full-time critics' strike me as generally sleazy and opportunistic, and I doubt the sincerity of their desire to 'improve' a field if all they can do is tear down others' work and not provide any of their own.

Actually, grown ups have bigger problem keeping this straigt. Grown ups made originally nazism happen after all. Grown ups dont want to insult nazi because of career or just trying to look nice and rational. Then again, young German soldiers were more cruel then old ones I read in historical accounts.

I don't know what you mean by eating grown up food here. There were quite a few unapologetic grown ups in history who were quite inflexible in their values.

Maybe it is another attempt to shame that has nothing to do with what actual grown ups do, for better or worst.

Fundamentally, however, I don't believe, for example, that physicists are more scrupulous/intelligent/honest than biologists, who are more so than psychologists. The irreproducibility rates in a field have to be primarily caused by the nature of the system studied in that field rather than some inherent attributes of the researchers.

So any field will have some constant level of irreproducibility which which essentially cannot be changed. But some researchers do better than others. My fundamental point is that if you want to change what can be changed, and genuinely improve a field, you have to focus the criticisms on specific bad actors rather than target an entire field at once and leverage a near-tautology (more complex systems yield more irreproducible results) into a 20-year tirade/career.

Also, helpfully for people like Ioannidis, a statement like "field X has lots of errors and irreproducibility" is non-falsifiable, and indeed will be true no matter what, so he is taking no risk with his criticisms -- even though they do more damage to public perception, morale, and funding, and help less to change things than targeted criticisms would do.

If I sound irritated by this sort of thing, it is because my graduate advisor dabbled in "meta-science" / "error correction" and I stoutly refused to have anything to do with those projects. I thought, and still think, that it is considerably more valuable to try to build my own solid lego tower despite the difficulties rather than knocking down other people's, despite the latter being considerably easier and a fairly simple and reliable route to a high citation count. In the end, irreproducible or erroneous papers in science just get ignored rather than retracted, and I think that's fine.

I have no idea if you’re right, and I’m not about to waste time down that particular rabbit hole. But you’re arguably doing wrong everything the article mentions. Starting with the impulse of “hold on, I’m not a crank and here are 12 paragraphs and my blog to prove it”. It’s simply off topic in this discussion.

https://medium.com/@jamesheathers/long-hair-dont-care-5eeba2...

Sample comments: If that title leads you to expect off-brand evo psych exploring massive behavioural changes induced through the complicated medium of hairdressing… well, you’d be right.

Mmmm.

It's also worth bearing in mind your audience.

This article was written by two guys who by and large are going after junk science in psychology, nutrition, and so on, where the worst motivation can only be to advance the careers of the scientists - and quite possibly the errors are just the result of over-optimism or lack of statistical abilities.

For that sort of work it makes sense to try and fix psychology from within, by pointing out errors. There isn't much point in trying to spread the word through the wider world that psychology is full of errors: what's the best outcome, exactly? A few people stop being invited to give TED talks? Big deal. Psychology isn't going to stop being studied, nor should it be!

But in the case of economics, there are two problems:

1) Economics is absolutely riddled with not just bad individual papers but entire sub-fields that are nonsense, it's an epidemic. It probably cannot be fixed by shaming individuals who make simple errors via their own journals and hoping for the field to improve. Even when major economists like Paul Krugman or Paul Romer have gone on very serious attacks against their own field, there has been no visible impact and in Romer's case he simply lost his job.

2) Economics is routinely used to justify massive government interventions (or lack of interventions) in markets and society. Bad economics research can have a very serious impact on people's lives.

In this case, it's probably better to work at a different level and try to spread the word amongst the general public that economics is not trustworthy. Thus the audience is not journals or the authors of the papers themselves. It's the public.

This is not hard because most people who follow the news have already encountered major cases where consensus economic predictions were simply wrong. Polling shows that trust in economists is quite low as is. If and when the field reforms it can try to rebuild trust.