All models are wrong, but some are completely wrong • Hacker News Board

Re: All models are wrong, but some are completely wrong

#101

Post by monocasa » Sun, Apr 05, 2020, 10:02 PM UTC

sah2ed wrote: post #90

rossdavidh wrote: post #56

So, I basically agree with everything this article says, but it seems to miss a basic point. If journalists do what this paper suggests, they make less money. Journalists, and the newsmedia corporations and organizations that employ them, don't run with the most inflammatory headline possible as an accidental fluke of a mistake that they were too careless to catch. Even public sector newsmedia organizations use measu…

> If there is one less to be learned from this whole Covid-19 debacle (and I'm sure there are several), it is that our entire news ecosystem, public and private, is fundamentally structured wrong for doing what is supposed to be its purpose, which is to make people better informed. It's not bad at it by mistake, it's bad at it as an inevitable consequence of its design. You speak of a failure in design as the root ca…

And particularly, as the concept applies to mass media, quite a few have noted that the purpose seems to be to create the consent of the populace for what the leadership wants to do.

Sometime's that's informing from a real benevolent perspective of let's get everyone on the same page for what's best to the best of our knowledge. Other times it's an obfuscation scheme to allow some to profit off of the rest of us.

https://en.wikipedia.org/wiki/Manufacturing_Consent

Re: All models are wrong, but some are completely wrong

#102

Post by a1369209993 » Sun, Apr 05, 2020, 10:02 PM UTC

heurifk wrote: post #22

puranjay wrote: post #10

I fail to understand how you can reasonably model an unprecedented event in modern history. We have no data on how people will act under a weeks, even months long lockdown. Will they stay indoors and follow guidelines? Maybe. Will they watch their livelihoods get affected, their mental health deteriorate, or get careless over time and break quarantine? Maybe. We just don't know because we just don't have the data or…

We have recent data from Wuhan. We have old data from 1918. This is a white swan, not a black one. More importantly, the mortality, while much higher than flu, it's still relatively low. Now imagine a virus as contagious as this one, but with 10% mortality over all age groups. That would be unprecedented and probably cause society meltdown.

> More importantly, the mortality, while much higher than flu, it's still relatively low.

Anecdote: someone was trying to convice me to panic about Coronavirus because "three hundred and [something] people died just today!"

  $ units
  8 billion / 80yr
  /day

"Actually, three hundred thousand people died today. Probably more, even."

Re: All models are wrong, but some are completely wrong

#103

Post by koyote » Sun, Apr 05, 2020, 10:18 PM UTC

a1369209993 wrote: post #102

heurifk wrote: post #22

Earlier quoted context omitted.

We have recent data from Wuhan. We have old data from 1918. This is a white swan, not a black one. More importantly, the mortality, while much higher than flu, it's still relatively low. Now imagine a virus as contagious as this one, but with 10% mortality over all age groups. That would be unprecedented and probably cause society meltdown.

> More importantly, the mortality, while much higher than flu, it's still relatively low. Anecdote: someone was trying to convice me to panic about Coronavirus because "three hundred and [something] people died just today!" $ units 8 billion / 80yr /day "Actually, three hundred thousand people died today. Probably more, even."

I think a lot of people would be very surprised if you told them how many people die in their country every year.

It's nearly 3 million in the US.

Re: All models are wrong, but some are completely wrong

#104

Post by DangitBobby » Sun, Apr 05, 2020, 10:23 PM UTC

hutzlibu wrote: post #84

JadeNB wrote: post #76

Earlier quoted context omitted.

> In academia, lying in a publication can be career-ending, which acts to broadly 'scare them straight'. For academics, it's desirable to be highly read and cited, but to be seen to be dishonest is the end of the road. Could it be possible to create similar incentives for journalists? The problem is that these incentives for academics aren't created out of purity of heart, but because of the system to which they belo…

Well, also as a journalist you can not get away with making up stories. There was a recent case of a high class journalist, who did that and was caught. https://en.wikipedia.org/wiki/Claas_Relotius But he did so very brutal so to say. The problem with journalism is mostly not straight lying, but missleading and bending the truth until it fits the agenda. So a classic journalist should not have another agenda than the…

What you're saying is probably broadly speaking true for journalists in general. But what many people actually consume news-wise can only be compared on the surface to actual journalism. Consumers of this material have no interest in determining if what they're reading is true, and there are no consequences for those spreading misinformation while pretending to be journalists. https://news.ycombinator.com/item?id=22784665

Re: All models are wrong, but some are completely wrong

#105

Post by dchyrdvh » Sun, Apr 05, 2020, 10:25 PM UTC

BiteCode_dev wrote: post #37

grawprog wrote: post #9

What I've noticed about models, or at least when people are talking about them or trying to prove a point about them, is that people forget models are a simplified version of a specific part of reality, much the same as models of airplanes or something. No matter how many variables you include, you can never capture the utterly massive and unpredictable amount of variables that exist in reality. But they're not suppo…

Humans have the tendency to think they understand something if they have a label on it. It helps them take decisions faster, whether it's good or bad. If you tell people you don't know what causes their respiratory illness, they may think you are incompetent, and feel helpless. If you tell them they have acute bronchitis, suddenly they feel empowered. It just means they have an inflammation of the bronchi. It says no…

"Mind is the great destroyer of reality" - a quote from one of those ancient dusty books. Our mind is a machine that creates models and labels for everything, to avoid dealing with the illigible reality.

Re: All models are wrong, but some are completely wrong

#106

Post by mr_toad » Sun, Apr 05, 2020, 10:35 PM UTC

monocasa wrote: post #101

sah2ed wrote: post #90

Earlier quoted context omitted.

> If there is one less to be learned from this whole Covid-19 debacle (and I'm sure there are several), it is that our entire news ecosystem, public and private, is fundamentally structured wrong for doing what is supposed to be its purpose, which is to make people better informed. It's not bad at it by mistake, it's bad at it as an inevitable consequence of its design. You speak of a failure in design as the root ca…

And particularly, as the concept applies to mass media, quite a few have noted that the purpose seems to be to create the consent of the populace for what the leadership wants to do. Sometime's that's informing from a real benevolent perspective of let's get everyone on the same page for what's best to the best of our knowledge. Other times it's an obfuscation scheme to allow some to profit off of the rest of us. htt…

> the purpose seems to be to create the consent of the populace for what the leadership wants to do.

Having spent a lot of time working for and with governments I can assure you that most of the time, behind closed doors, they truely hate the press, and wish it would go away. And this is true of even the most open and democratic governments.

Autocratic governments, again and again have shown what they think of a free press.

The idea that mainstream media is a mouthpiece of government just doesn’t add up.

Re: All models are wrong, but some are completely wrong

#107

Post by schnable » Sun, Apr 05, 2020, 10:36 PM UTC

grawprog wrote: post #9

What I've noticed about models, or at least when people are talking about them or trying to prove a point about them, is that people forget models are a simplified version of a specific part of reality, much the same as models of airplanes or something. No matter how many variables you include, you can never capture the utterly massive and unpredictable amount of variables that exist in reality. But they're not suppo…

Which leads to the saying that this headline is a play on: "All models are wrong, but some are useful." (George E.P. Box)

We probably should be repeating that a lot to make sure people hear it.

Re: All models are wrong, but some are completely wrong

#108

Post by SpicyLemonZest » Sun, Apr 05, 2020, 10:37 PM UTC

kick wrote: post #93

chrischattin wrote: post #75

Earlier quoted context omitted.

This sounds good in an edgy r/im14andthisisdeep sort of way. But in reality, having integrity and ethics in business is actually more profitable over the long term.

The two richest people on the planet right now are both known for being almost entirely void of morals when it comes to their business dealings.

They're known for this because both activists and entrenched business interests benefit from promoting that narrative. Gates and Bezos both regularly talk about the impact they have on the world and why that matters more than just money - there are reasonable arguments that they've done bad things, but the idea that they're completely amoral voids is just silly.

Re: All models are wrong, but some are completely wrong

#109

Post by kerkeslager » Sun, Apr 05, 2020, 10:37 PM UTC

sah2ed wrote: post #90

rossdavidh wrote: post #56

So, I basically agree with everything this article says, but it seems to miss a basic point. If journalists do what this paper suggests, they make less money. Journalists, and the newsmedia corporations and organizations that employ them, don't run with the most inflammatory headline possible as an accidental fluke of a mistake that they were too careless to catch. Even public sector newsmedia organizations use measu…

> If there is one less to be learned from this whole Covid-19 debacle (and I'm sure there are several), it is that our entire news ecosystem, public and private, is fundamentally structured wrong for doing what is supposed to be its purpose, which is to make people better informed. It's not bad at it by mistake, it's bad at it as an inevitable consequence of its design. You speak of a failure in design as the root ca…

If you're smart enough to come up with a point like this, why not use that intelligence to make a less inane point? You're not wrong, you're just correct in a way that responds to a very narrow reading of what your parent comment was saying, and doesn't make any attempt to figure out why they'd be saying it.

Clearly the person you're responding to wants a system that makes people better informed, and I think there's a lot of other people (myself included) who want the same thing. So the question is obviously, how do we change the news ecosystem from what it is, to a system that makes people better informed?

Re: All models are wrong, but some are completely wrong

#110

Post by gpm » Sun, Apr 05, 2020, 10:38 PM UTC

sgustard wrote: post #89

cousin_it wrote: post #68

> Journalists must get quotes from other experts before publishing No, this isn't enough. This whole way of thinking isn't enough. It's a big part of the reason for the current situation. Journalists should report what's true, not what Tom, Dick or Harry said. If a journalist isn't qualified to make object-level claims on a given topic, don't write on that topic. For example, if Bob says there's a forest fire, then i…

Um, so if you want to publish an article on how many people will die of the virus, how do you ensure it's true? Obviously you either don't write the article, or you quote the most reliable projections you can find. But those are still projections.

You can't honestly write "an article on how many people will die of the virus" which gives a number. No one knows.

What you can do is write an article about projections of how many people will die. In it you talk about the data sources being used in the model(s), the assumptions being made in the model(s), and the result they give including likely major sources of error..

You can put more or less detail depending on your audience. But it should always include enough detail that your audience understands that it is an estimate based on assumptions and likely flawed data, and you should always understand the model you are writing about even if you don't explain it.