Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

All models are wrong, but some are completely wrong

rssdss.design.blog

Post reply on HN
Print view View original

11–20 of 216 posts

Re: All models are wrong, but some are completely wrong

#11

Post by ajuc »

puranjay wrote: post #10

I fail to understand how you can reasonably model an unprecedented event in modern history. We have no data on how people will act under a weeks, even months long lockdown. Will they stay indoors and follow guidelines? Maybe. Will they watch their livelihoods get affected, their mental health deteriorate, or get careless over time and break quarantine? Maybe. We just don't know because we just don't have the data or…

There's very little unprecedented events in modern history, and pandemic certainly isn't one.

Re: All models are wrong, but some are completely wrong

#12

Post by dwohnitmok »

martingoodson wrote: post #7
dwohnitmok wrote: post #6

Earlier quoted context omitted.

This is a hard problem, mainly because of the problem of motivated reasoning as mentioned by another comment. You're requiring cooperation from multiple different parties here (scientists, journalists, policy makers, readers, etc.) and any of these parties can warp the results in any number of ways regardless of the cooperation of other parties. Climate science still hasn't solved this problem despite trying to imple…

I think, at the least, if journalists get some quotes from other scientists before publishing a piece on a new model it would be a major win.

That's fair. I mean asking folks to be more circumspect in general is probably a good thing.

I prefer though to look at problems through the lens of incentive structures (keeping in mind humans generally heavily time discount incentives and what incentivizes people is not always obvious! Death isn't always much of a disincentive beyond a rather short time horizon). And here I'm having a hard time seeing easy ways to tweak the incentive structure.

Re: All models are wrong, but some are completely wrong

#13

Post by thaumasiotes »

martingoodson wrote: post #7
dwohnitmok wrote: post #6

Earlier quoted context omitted.

This is a hard problem, mainly because of the problem of motivated reasoning as mentioned by another comment. You're requiring cooperation from multiple different parties here (scientists, journalists, policy makers, readers, etc.) and any of these parties can warp the results in any number of ways regardless of the cooperation of other parties. Climate science still hasn't solved this problem despite trying to imple…

I think, at the least, if journalists get some quotes from other scientists before publishing a piece on a new model it would be a major win.

Often they do, but that doesn't affect the piece they write. It's not at all difficult to find people complaining "I was interviewed for this article, most of what I said was left out, and to the extent I am quoted, it's to give the impression that my beliefs are the exact opposite of what I explained to the journalist at length".

Re: All models are wrong, but some are completely wrong

#14

Post by YZF »

martingoodson wrote: post #3

Author here: happy to take comments or criticism

I'm finding myself in disagreement with rule #6. Using a model effectively is about a lot more than just the domain knowledge. I'd value analysis from a mathematician/statistician more highly than from an infectious disease physician. There's the stuff that informs models, i.e. the observations, the experimentation etc. and then there's the science of modelling itself which isn't really in the same domain.

Re: All models are wrong, but some are completely wrong

#15

Post by martingoodson »

puranjay wrote: post #10

I fail to understand how you can reasonably model an unprecedented event in modern history. We have no data on how people will act under a weeks, even months long lockdown. Will they stay indoors and follow guidelines? Maybe. Will they watch their livelihoods get affected, their mental health deteriorate, or get careless over time and break quarantine? Maybe. We just don't know because we just don't have the data or…

If you're interested in this specific area, here is a useful summary of what's known: https://www.thelancet.com/journals/lancet/article/PIIS0140-6...

tl;dr not much

Re: All models are wrong, but some are completely wrong

#16

Post by martingoodson »

YZF wrote: post #14
martingoodson wrote: post #3

Author here: happy to take comments or criticism

I'm finding myself in disagreement with rule #6. Using a model effectively is about a lot more than just the domain knowledge. I'd value analysis from a mathematician/statistician more highly than from an infectious disease physician. There's the stuff that informs models, i.e. the observations, the experimentation etc. and then there's the science of modelling itself which isn't really in the same domain.

I agree with this - but I do think it should be clear that the model is from outside the mainstream. Not to dismiss it but to clarify its status. Check out the New Yorker piece I link to in the article - it's quite shocking the misinformation that's out there.

Re: All models are wrong, but some are completely wrong

#19

Post by excalibur »

martingoodson wrote: post #7
dwohnitmok wrote: post #6

Earlier quoted context omitted.

This is a hard problem, mainly because of the problem of motivated reasoning as mentioned by another comment. You're requiring cooperation from multiple different parties here (scientists, journalists, policy makers, readers, etc.) and any of these parties can warp the results in any number of ways regardless of the cooperation of other parties. Climate science still hasn't solved this problem despite trying to imple…

I think, at the least, if journalists get some quotes from other scientists before publishing a piece on a new model it would be a major win.

Asking more of journalists at this juncture may be a tall order. The bar for them has been trending downward, due largely to a lack of funding and an abundance of competition, many of whom have little regard for journalistic standards.

Re: All models are wrong, but some are completely wrong

#20

Post by veddox »

puranjay wrote: post #10

I fail to understand how you can reasonably model an unprecedented event in modern history. We have no data on how people will act under a weeks, even months long lockdown. Will they stay indoors and follow guidelines? Maybe. Will they watch their livelihoods get affected, their mental health deteriorate, or get careless over time and break quarantine? Maybe. We just don't know because we just don't have the data or…

On the contrary, epidemics are reasonably easy to model (at least at a basic level), and modelling them has a long academic tradition. As with all modelling, it all depends on what kind of detail you want to get out of your analysis/predictions.

If you‘re looking at a simple population infection model, lockdown efficacy is just a factor affecting the contact rate among uninfected actors. You run multiple scenarios for multiple levels of this factor, and see where that gets you.

Or did you mean something else?

(Source: I‘m not an epidemiologist, but as an ecological modeller I work with very similar tools.)

Post reply on HN