On “Armchair Epidemiology”

If you look at the stuff coming out of Imperial and Oxford in recent weeks, there are a huge number of basic problems. Their papers can't be replicated for multiple reasons. They're using known-bad input data. They're not providing uncertainty bounds. There's no peer review. The Imperial paper assumed constant hospital capacity. Their papers reach diametrically opposite conclusions. They have a track record of model failure.

It's all practically a poster child for the replication crisis.

The /r/physics post criticises physicists who write some code to do basic curve fitting. Has he looked at the Oxford epidemiology paper? It is by all accounts literally curve fitting to the first 15 days of outbreak and reaches diametrically opposite conclusions to the Imperial paper.

It criticises people who write papers and then upload them to arXiv because reporters will find it and create panic. Would this person prefer people to use the Imperial/Oxford technique of sending papers directly to journalists, and entirely skipping the whole upload to arXiv step?

It blames physicists for creating "denial": "You become a punchline to a denier that says, they can't decide if there's going to be hundred thousand cases or a hundred million cases! Scientists don't know anything!" - guess what, they're saying that already and are going to say it a lot more because epidemiologists themselves keep contradicting and criticising each other for doing bad work, in public. Additionally, there are enough people with scientific literacy to understand the limits of statistics and modelling out there. They aren't as easily confused as this person rather seems to assume (does he have any data showing that people conflate epidemiology with physics?).

Right now I absolutely want to see papers written by physicists studying COVID-19. Why - because physics is a significantly more rigorous field than epidemiology. I trust the average physicist to have at least slightly higher standards, for instance I trust them to at least pretend to care about statistical uncertainty, and I suspect many of them will upload their source code. I don't expect any of them to email their paper straight to known-friendly newspapers.

What's so hard about masks? That's one area that experts seem pretty consistent on, and what they say makes sense. It is slightly complicated but not contradictory.

Wearing a mask doesn't do that much to prevent you from getting sick. But it does a lot to prevent you from getting others sick.

If there is a limited supply of masks, the best people to give them to are people are at high likelihood of being sick and come face to face with lots of people who get sick easily. Which is to say medical staff.

If there were masks to go around, we'd want everyone wearing them as a precaution. So that you're not getting people sick even if you don't yet know that you are sick.

Some countries, like Taiwan, already make enough masks that they can actually do that. The USA does not. In time we should fix that.

Can you find anyone whose expertise is generally respected who says something that disagrees with what I just said?

First of all, I think calling this a single event is reductionist. We're talking about weeks- months, even- of experts and non experts warning about this, and the systems of authorities failed completely to react in time. We're not talking about failing to heed an important warning once, we're talking about a long string of choices here.

Secondly, This isn't just a prominent case, it is a very important case. Treating this as a series of bernoulli trials with a single (statistically irrelevant) failure is an extremely reductionist model of the reality here.

While your conclusion of trusting science, empirical facts and analysis rings true to me, I think we should be wary of overly simplifying the subject matter.

A standard of "scientists dabbling in journalism to improve communication" instead of "journalists dabbling in communicating science" would likely have gone a long way to improving the civilian response and trust in experts.

There's also the factor of the "Noble Lie" dishonesty around wearing masks. I suspect intellectual-ish contrarians (like Aaronson et al) are more angered by this than the average person, but we ended up with shortages and sellouts anyways: a significant number of non-experts didn't buy into that noble lie so the experts/organizations that pushed it (not to be confused with ALL experts) burned a lot of good will to basically no effect.

ETA: I considered replying to your comment about the Harvard epidemiologists but would rather edit it in here to avoid two replies to one person: I think that's a great example of my point. The story wasn't broken by bloggers, but bloggers were more likely to be amplifying the concerned experts than our "traditional media powerhouses."