All models are wrong, but some are completely wrong • Hacker News Board

Re: All models are wrong, but some are completely wrong

#111

Post by hinkley » Sun, Apr 05, 2020, 10:41 PM UTC

darkerside wrote: post #52

tartoran wrote: post #48

Earlier quoted context omitted.

The low hanging fruit metaphor meaning do the simplest or easiest work first is not really applicable to fruit growing but seems to have been coopted by the busieness world instead, then we started using it too in our contexts. Not crazy about this metaphor either, but it could be useful in some contexts and while also misunderstood in others. The same goes for most jargon.

Low hanging fruit is a point in time tactic, not a generalized recommendation

And when you do that for a very long time they call it short-term thinking. Myopia. Missing the forest for the trees.

Plenty of that going around.

Re: All models are wrong, but some are completely wrong

#112

Post by m0zg » Sun, Apr 05, 2020, 10:43 PM UTC

I'd like to point out that we _still_ don't know if that "high R0" model was right or not. And we won't know that until we randomly test a sufficiently large random sample of the UK population for antibodies. They imposed containment _very_ late, and they go to pubs _all the time_. It is not implausible that the majority of their population already had COVID19 without even knowing what it was.

What we do know is that "doomsday style" models from IHME that just last week were predicting 50K beds needed in NY are off by a factor of 3-4, and hospitalization are starting to flatten out already. You can guess the direction they were wrong in. And before you start an uninformed argument, yes, these models assume the current isolation measures.

In the meanwhile NY hoarded the ventilators and medical supplies because it anticipated this prediction to be true. To be clear, I don't blame NY - they used the best information they had, which turned out to be bullshit. Better be safe than sorry.

These are not harmless errors. When this is over, someone should study these fiascos and estimate the death toll just from bad models alone.

https://twitter.com/AlexBerenson/status/1246465515704463360

Re: All models are wrong, but some are completely wrong

#113

Post by em500 » Sun, Apr 05, 2020, 10:49 PM UTC

hutzlibu wrote: post #84

JadeNB wrote: post #76

Earlier quoted context omitted.

> In academia, lying in a publication can be career-ending, which acts to broadly 'scare them straight'. For academics, it's desirable to be highly read and cited, but to be seen to be dishonest is the end of the road. Could it be possible to create similar incentives for journalists? The problem is that these incentives for academics aren't created out of purity of heart, but because of the system to which they belo…

Well, also as a journalist you can not get away with making up stories. There was a recent case of a high class journalist, who did that and was caught. https://en.wikipedia.org/wiki/Claas_Relotius But he did so very brutal so to say. The problem with journalism is mostly not straight lying, but missleading and bending the truth until it fits the agenda. So a classic journalist should not have another agenda than the…

Best we can tell, Bloomberg's Chinese Big Hack story was completly made up, but it doesn't appear anyone involved suffered any consequences.

https://appleinsider.com/articles/19/10/04/editorial-a-year-...

Re: All models are wrong, but some are completely wrong

#114

Post by sah2ed » Sun, Apr 05, 2020, 10:50 PM UTC

perl4ever wrote: post #96

sah2ed wrote: post #90

Earlier quoted context omitted.

> If there is one less to be learned from this whole Covid-19 debacle (and I'm sure there are several), it is that our entire news ecosystem, public and private, is fundamentally structured wrong for doing what is supposed to be its purpose, which is to make people better informed. It's not bad at it by mistake, it's bad at it as an inevitable consequence of its design. You speak of a failure in design as the root ca…

"From a teleological perspective, viewing the media as a tool designed “to inform” would be a mistake, it would amount to nothing more than a supposition." I and perhaps many other people think the media should inform people; that is a statement of its purpose, and it appears that the collective actions of people in society have redesigned it to eliminate most of the informing. It doesn't have to have ever been desig…

> I and perhaps many other people think the media should inform people; that is a statement of its purpose, and it appears that the collective actions of people in society have redesigned it to eliminate most of the informing.

Your expectation that the media should inform people is reasonable but unrealistic. The original designers had fairly specific design goals in mind and it did not explicitly include “to inform”, this can only be tacked on.

Today we have newspapers, radio, TV and of course the Internet as tools of (mass) media.

If you were among the original designers of the newspaper[0], which was the first tool of mass media for centuries, but your preference to inform was out-weighed by competing interests (aka politics), then of course you are well within your rights to be outraged — you are entitled to criticize the design for falling short of your expectations.

In other words, unless you were personally part of the design team, you can’t really speak about what the design should be or should have been, since you are in no position to influence the system’s purpose before it was built.

You can only deal with the consequences of the designers’ creation, which includes working around the limitations of the system as designed, or building a new system from scratch.

A new system that addresses those limitations would not only be expensive to build from scratch, it would also have to compete for attention and funding to be sustainable — the market has to decide if they truly care about being better informed or not.

Facebook, which is perhaps the most modern type of media, is as vulnerable as the others when it comes to ability to misinform.

[0] ”News was highly selective and often propagandistic. Readers were eager for sensationalism, such as accounts of magic, public executions and disasters; this material did not pose a threat to the state, because it did not pose criticism of the state.” culled from Wikipedia: https://en.wikipedia.org/wiki/History_of_newspaper_publishin...

Re: All models are wrong, but some are completely wrong

#115

Post by monocasa » Sun, Apr 05, 2020, 10:51 PM UTC

mr_toad wrote: post #106

monocasa wrote: post #101

Earlier quoted context omitted.

And particularly, as the concept applies to mass media, quite a few have noted that the purpose seems to be to create the consent of the populace for what the leadership wants to do. Sometime's that's informing from a real benevolent perspective of let's get everyone on the same page for what's best to the best of our knowledge. Other times it's an obfuscation scheme to allow some to profit off of the rest of us. htt…

> the purpose seems to be to create the consent of the populace for what the leadership wants to do. Having spent a lot of time working for and with governments I can assure you that most of the time, behind closed doors, they truely hate the press, and wish it would go away. And this is true of even the most open and democratic governments. Autocratic governments, again and again have shown what they think of a free…

That's not an issue unless you think of leadership as a homogeneous blob. Of course they don't like the portion of the media pushing the other guy's cognitive structures; and ultimately that's what they mean when they complain about the media.

The way you can particularly see this is in the suppression of news that doesn't ultimately help out _any_ of the current leaders with it's dissemination, only the populace.

Re: All models are wrong, but some are completely wrong

#116

Post by smitty1e » Sun, Apr 05, 2020, 10:52 PM UTC

thaumasiotes wrote: post #100

YZF wrote: post #21

Earlier quoted context omitted.

This is driving me crazy ;) the poor quality of data. You'd want health organizations around the world to be publishing every possible detail (anonymized) so that the disease can be better understood. Yet three months in, with over a million cases worldwide, we still have experts disagreeing about things like asymptomatic transmission, use of masks, droplets vs. aerosol, how much distance one should stand from anothe…

> You'd want health organizations around the world to be publishing every possible detail (anonymized) If you're publishing every possible detail, then your data isn't anonymized. Anonymization consists of the removal of almost all of the details.

Or even if the data are reasonably tidy, they might expose more when juxtaposed with another dataset.

Re: All models are wrong, but some are completely wrong

#117

Post by Viliam1234 » Sun, Apr 05, 2020, 10:54 PM UTC

renewiltord wrote: post #74

rossdavidh wrote: post #56

So, I basically agree with everything this article says, but it seems to miss a basic point. If journalists do what this paper suggests, they make less money. Journalists, and the newsmedia corporations and organizations that employ them, don't run with the most inflammatory headline possible as an accidental fluke of a mistake that they were too careless to catch. Even public sector newsmedia organizations use measu…

The problem is on the demand side, not solely on the supply side. People desire reinforcement not challenge. That is natural. I suspect it is true of us all. Those who are capable direct their potential to be challenged in specific ways, allowing themselves to have biases reinforced in the unimportant places. And they have access to the knowledge in their specific areas. This is the reason for the enduring power of t…

The problem is also in the change of how the demand can manifest. For example, I would never buy a clickbait newspaper. But sometimes I click on clickbait articles online. In both cases, there is a part of me that is interested, and a part of me that knows better. If it only requires a mouse click, the interested part sometimes wins. If it requires taking out my purse, the interested part loses this fight.

The horrible thing about online advertising is that it allows people to make profit from making you look at something, even if it immediately makes you disappointed. From that moment, the need to write non-disappointing articles has decreased significantly.

Re: All models are wrong, but some are completely wrong

#118

Post by TwoBit » Sun, Apr 05, 2020, 10:54 PM UTC

martingoodson wrote: post #7

dwohnitmok wrote: post #6

Earlier quoted context omitted.

This is a hard problem, mainly because of the problem of motivated reasoning as mentioned by another comment. You're requiring cooperation from multiple different parties here (scientists, journalists, policy makers, readers, etc.) and any of these parties can warp the results in any number of ways regardless of the cooperation of other parties. Climate science still hasn't solved this problem despite trying to imple…

I think, at the least, if journalists get some quotes from other scientists before publishing a piece on a new model it would be a major win.

Journalists and their publications don't give a crap about scientific truth. They care about maximizing clicks and traffic. If a group of scientists give them the truth and another group gives them a more sensational story, they'll print the latter. Scientists will never win this battle.

Re: All models are wrong, but some are completely wrong

#119

Post by amitdeshwar » Sun, Apr 05, 2020, 10:57 PM UTC

renewiltord wrote: post #74

rossdavidh wrote: post #56

So, I basically agree with everything this article says, but it seems to miss a basic point. If journalists do what this paper suggests, they make less money. Journalists, and the newsmedia corporations and organizations that employ them, don't run with the most inflammatory headline possible as an accidental fluke of a mistake that they were too careless to catch. Even public sector newsmedia organizations use measu…

The problem is on the demand side, not solely on the supply side. People desire reinforcement not challenge. That is natural. I suspect it is true of us all. Those who are capable direct their potential to be challenged in specific ways, allowing themselves to have biases reinforced in the unimportant places. And they have access to the knowledge in their specific areas. This is the reason for the enduring power of t…

What about 538?

Re: All models are wrong, but some are completely wrong

#120

Post by salty_biscuits » Sun, Apr 05, 2020, 11:00 PM UTC

nitrogen wrote: post #88

Would we encourage an epidemiologist to apply ‘fresh thinking’ to the design of an electrical substation? Yes, absolutely. If an epidemiologist identifies and models a trend in human disease around substations, or a trend in failures of substations, or a new way of modeling the ways electrical demand can change over time and influence demand in other times and places, etc., then their input should absolutely be consi…

That rubbed me up the wrong way too. Applied mathematics is really quite transferable across disciplines, you just usually don't have the domain knowledge to stop you chasing dead ends or reinventing the wheel.