Earlier quoted context omitted.
I think, at the least, if journalists get some quotes from other scientists before publishing a piece on a new model it would be a major win.
I don’t think there’s value in doing this, unless the second opinion disagrees with the original model. It’s not hard to find a second “expert” to agree with just about anything, if you look hard enough. The number of people who agree with something tells you, as a rational person, practically nothing - it reminds me of the absurd compilation argument “One Hundred Authors Against Einstein”: https://archive.org/detail…
All models are wrong, but some are completely wrong
171–180 of 216 posts
Re: All models are wrong, but some are completely wrong
#172Earlier quoted context omitted.
It's easy to lie without lying. As a society we replaced any sort of value for our value proxy, money. So truth ethics etc. are all secondary values.
This sounds good in an edgy r/im14andthisisdeep sort of way. But in reality, having integrity and ethics in business is actually more profitable over the long term.
More generally, plenty of companies cause negative externalities, profiting by causing damage that they don't have to pay for.
Re: All models are wrong, but some are completely wrong
#173I kind of disagree with this point. The models we see are mostly statistical models. Anybody with a statistics background (mathematician, physicist, chemist, biologist, engineer, computer scientist, epidemiologist) can have enough of an understanding to make valid points.
Discarding opinions based on somebody's background is an argumentative fallacy in and of itself. You should of course check if somebody is trustworthy, but epidemiologists should be scrutinized under this aspect like everybody else.
Re: All models are wrong, but some are completely wrong
#174Earlier quoted context omitted.
> While saying that "chloroquine phosphate is a good prophylactic for covid-19" to someone who would believe it certainly should be punishable (as attempted murder perhaps! and regardless of whether the speaker is a licensed physician!). I would disagree. As an example in the history, saying "The Earth is a sphere" was once punishable by death. Fake news and pseudo science has always existed, but censorship is not a…
Chloroquine phosphate is fishtank cleaner, and -to humans- poison. Granted, if you're just dumb and ignorant and choose to take it yourself, that's no crime. But telling others to take poison, without telling them it's poison, should be a crime. Of course, in the case of chloroquine phosphate, if you look at the packaging, you'll see it's poison.
Chloroquine phosphate is a fish tank cleaner. On the other hand, chloroquine phosphate is a medicine.
Re: All models are wrong, but some are completely wrong
#175> Journalists must get quotes from other experts before publishing No, this isn't enough. This whole way of thinking isn't enough. It's a big part of the reason for the current situation. Journalists should report what's true, not what Tom, Dick or Harry said. If a journalist isn't qualified to make object-level claims on a given topic, don't write on that topic. For example, if Bob says there's a forest fire, then i…
> Journalists should report what is true, not what Tom, Dick or Harry said. Especially they should not give 33% plausibility to Tom, 33% to Dick and 33% to Harry. Which is what they typically do, and call that "professional journalism." If Tom represents 95% of scientists and Dick and Harry the fringe 5% also financed by (let say) tobacco industry or oil corporations, or Boeing, or those paid by the CIA, they should…
What actually happens is this:
1. Journalists have a story they want to write. They probably already decided what the message is going to be, but let's be generous and assume they didn't.
2. They consult a rolodex of, almost exclusively, government funded academics. This is true even if the story they're writing is about activity or science that takes place only in the private sector and for which the academics in question have no actual experience. They may consult two or three grant funded academics if they want their story to seem especially robust. If they give the private sector a chance to reply at all (often not) they will cite one or two sentences of a brief phone call in which the person being talked to doesn't know what they're trying to defend themselves against and is probably just confused. It's not a real interview with an actual in-house expert.
3. This story is then published as "Experts say, ..." even if what the chosen experts say flatly contradicts common sense or things that can be checked with 10 minutes and a search engine.
4. Readers comment below the line, pointing out the flaws in the story. If a specific company is involved, they may do a blog post explaining their side of the story which the journalists will either completely ignore, or if they think they can get away with it, selectively quote one or two sentences in a misleading way.
Industrial scientists/engineers are sometimes assumed to be inherently evil and untrustworthy. But the whole idea that journalists are untrusted because they very rarely quote people in industry is a saw of the left; it's not true, which is why the examples given always seem so curiously weird and out of date. Tobacco industry? when was the last time you even read a newspaper article about them? Oil corporations? Those firms that have spent the last 15 years rebranding themselves as energy companies because they now make solar panels too?
There's a lot of really good analysis of the trouble journalism has got itself in for, and a significant part of the blame is laid at the feet of journalists uncritically reporting anything academics say as the Whole Truth and Nothing But The Truth, when in fact they routinely contradict each other, make up statistics, report obvious common sense as "findings", are hugely over-confident in their own predictive abilities, mis-use statistics and so on.
A good book to read on the topic is "Wrong" by David Freedman:
https://www.goodreads.com/book/show/8134625-wrong
He's a former journalist and so has direct experience of this problem. He cites many examples where expert testimony caused misleading or wrong stories to be published, but IIRC none of them involved corporate scientists. Mostly academics like nutritionists, psychologists and so on.
Re: All models are wrong, but some are completely wrong
#176I am working on trying to make an accurate model for predicting COVID-19 growth based on the per-county figures we have with COVID-19 (courtesy The New York Times). To say the data is noisy would be an understatement. What I have found, so far, is that if we look at current daily growth (averaged over seven days) and use exponentiation to predict future growth based on the previous week’s figures, the numbers are too…
https://docs.google.com/spreadsheets/d/1PqMVPU0VYcDWrUnDh1Cs...
Basically, it uses the fact that mean time from contagion to death is 17,3 days. For France, it gives reasonably accurate predictions. Best than almost any other model I've seen, in fact.
Re: All models are wrong, but some are completely wrong
#177Earlier quoted context omitted.
I'm finding myself in disagreement with rule #6. Using a model effectively is about a lot more than just the domain knowledge. I'd value analysis from a mathematician/statistician more highly than from an infectious disease physician. There's the stuff that informs models, i.e. the observations, the experimentation etc. and then there's the science of modelling itself which isn't really in the same domain.
I agree with this - but I do think it should be clear that the model is from outside the mainstream. Not to dismiss it but to clarify its status. Check out the New Yorker piece I link to in the article - it's quite shocking the misinformation that's out there.
Which model? Your article mostly dwells on the Oxford model in the FT which is as mainstream as it gets. The Daily Mail article is about a prediction by an Imperial academic.
There doesn't appear to be any "mainstream" in epidemiology. This kind of false-consensus argument is one I've seen before and frankly it undermines people's respect in both academic and journalism. How exactly is the "mainstream" established, in your mind? And don't talk about vague, easily manipulated non-metrics like reputation. Mainstream models should come from a single thing: success. Your insistence on rigid academic segmentation is the very thing that makes academia brittle and riven with non-replicable papers.
You know which kind of modellers I respect the most? Actuaries. They have skin in the game. They don't try to predict pandemics or sell insurance products against them, implying they think pandemics can't be modelled. So far based on what I've read about the history of epidemiology, I can't disagree with them.
Re: All models are wrong, but some are completely wrong
#178"the best material model of a cat is another, or preferably the same, cat” - Norbert Weiner
Re: All models are wrong, but some are completely wrong
#179Earlier quoted context omitted.
This is a hard problem, mainly because of the problem of motivated reasoning as mentioned by another comment. You're requiring cooperation from multiple different parties here (scientists, journalists, policy makers, readers, etc.) and any of these parties can warp the results in any number of ways regardless of the cooperation of other parties. Climate science still hasn't solved this problem despite trying to imple…
I think, at the least, if journalists get some quotes from other scientists before publishing a piece on a new model it would be a major win.
Which is fine. It'd be far better if people weren't trying to model COVID-19 at all. It's clearly far beyond our societies capabilities today. We can't even get useful data let alone feed it into a simulation of society.
Re: All models are wrong, but some are completely wrong
#180Earlier quoted context omitted.
> Journalists should report what is true, not what Tom, Dick or Harry said. Especially they should not give 33% plausibility to Tom, 33% to Dick and 33% to Harry. Which is what they typically do, and call that "professional journalism." If Tom represents 95% of scientists and Dick and Harry the fringe 5% also financed by (let say) tobacco industry or oil corporations, or Boeing, or those paid by the CIA, they should…
What you describe here is a deeply problematic fallacy and in fact what journalists already do. They are very, very far from your hypothetical "33% to Tom, 33% to Dick, 33% to Harry" scenario and it's one of the factors that undermines trust in journalism. What actually happens is this: 1. Journalists have a story they want to write. They probably already decided what the message is going to be, but let's be generous…
So it's completely unrelated to the topics I refer to. What I refer to is:
https://www.amazon.com/Merchants-Doubt-Handful-Scientists-Ob...
Claiming that the misinterpretation of science happened only in the distant past is intentional attempt to obscure the real problem.
There is objective truth and it is far from what some people with a lot of money peddle as the truth and what gets replicated across the media. And the media definitely don't cover what effectively advertising campaigns are as such -- paid disinformation for the benefits of some specific corporations or interest groups.