Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Put not thy trust in Nate Silver

thenewatlantis.com

Post reply on HN
Print view View original

21–26 of 26 posts

Re: Put not thy trust in Nate Silver

#21

Post by bryanlarsen »

borramakot wrote: post #15
borramakot wrote: post #13

Earlier quoted context omitted.

Boy, have I got the incredibly specific article for that exact question: https://fivethirtyeight.com/features/when-we-say-70-percent-...

I don't think this article provides strong evidence that they are well calibrated on the presidential election specifically (sample size N=3), or that they are correctly accounting for rare black swan events, but it does seem to imply that the criticisms about "538 claims victory no matter what because they always have non-zero probabilities" are oversimplified.

2016 wasn't a black swan event. It was a polling error, which do happen, if rarely. It was not unforseeable, and 538 included a probability of that happening which is why they gave Trump higher chances than most others did.

And 538 does do backtesting on elections back to 1972. That's not particularly trustworthy since it invites over-fitting, but internally they do have a little bit more than N=3 to work from.

Re: Put not thy trust in Nate Silver

#22

Post by xlm1717 »

scarmig wrote: post #6

It's pretty lame to paste an image of the NY Time's 82% chance of Clinton winning the election in an article that calls out Nate by name. Nate gave Clinton like a 70% chance. Can model building predict human behavior? It's fuzzier than the hard sciences, but not that much fuzzier. And if you do believe it's impossible to model human behavior, you have to throw out vast swaths of knowledge and areas of study: there go…

Economics is being thrown out in favor of Modern Monetary Theory.

Psychology has a reproducibility crisis.

Epidemiology... isn't doing so well.

And I have no problem throwing out sociology and anthropology.

Re: Put not thy trust in Nate Silver

#23

Post by scarmig »

xlm1717 wrote: post #22
scarmig wrote: post #6

It's pretty lame to paste an image of the NY Time's 82% chance of Clinton winning the election in an article that calls out Nate by name. Nate gave Clinton like a 70% chance. Can model building predict human behavior? It's fuzzier than the hard sciences, but not that much fuzzier. And if you do believe it's impossible to model human behavior, you have to throw out vast swaths of knowledge and areas of study: there go…

Economics is being thrown out in favor of Modern Monetary Theory. Psychology has a reproducibility crisis. Epidemiology... isn't doing so well. And I have no problem throwing out sociology and anthropology.

All of these wildly miss the point.

> Economics is being thrown out in favor of Modern Monetary Theory.

MMT is many things, but it's most definitely not a rejection of the idea that human behavior can be modeled.

Re: Put not thy trust in Nate Silver

#24

Post by dave333 »

xhkkffbf wrote: post #2

I think Nate is pretty sharp -- but I don't trust the people who answer polls. Who has time to pick up the phone? Most of the time it's some spammer. And if you do answer, why tell the truth about your vote?

This what scares me about the polls - that they don't account for Trump supporters being less truthful than the rest of us. Just look at Trump - he hasn't said anything true for years.

Re: Put not thy trust in Nate Silver

#25

Post by borramakot »

bryanlarsen wrote: post #21
borramakot wrote: post #15

Earlier quoted context omitted.

I don't think this article provides strong evidence that they are well calibrated on the presidential election specifically (sample size N=3), or that they are correctly accounting for rare black swan events, but it does seem to imply that the criticisms about "538 claims victory no matter what because they always have non-zero probabilities" are oversimplified.

2016 wasn't a black swan event. It was a polling error, which do happen, if rarely. It was not unforseeable, and 538 included a probability of that happening which is why they gave Trump higher chances than most others did. And 538 does do backtesting on elections back to 1972. That's not particularly trustworthy since it invites over-fitting, but internally they do have a little bit more than N=3 to work from.

(I have fairly minor quibbles with some of Nate's modeling ideas, but I broadly mean to be defending him).

I don't mean to imply 2016 was a black swan event- I agree that ~30% was probably as accurate a take as could be achieved (most evidence that seems reasonable to use indicated a lead for Clinton, but that it wouldn't be that surprising for that lead to be overcome). I just mean that the model assumes a fairly normal election environment, without like a huge attack on Election day or something on election day.

The N=3 comment was meant specifically for evaluating their calibration, not the data they use for their model.

Re: Put not thy trust in Nate Silver

#26

Post by dwaltrip »

xlm1717 wrote: post #22
scarmig wrote: post #6

It's pretty lame to paste an image of the NY Time's 82% chance of Clinton winning the election in an article that calls out Nate by name. Nate gave Clinton like a 70% chance. Can model building predict human behavior? It's fuzzier than the hard sciences, but not that much fuzzier. And if you do believe it's impossible to model human behavior, you have to throw out vast swaths of knowledge and areas of study: there go…

Economics is being thrown out in favor of Modern Monetary Theory. Psychology has a reproducibility crisis. Epidemiology... isn't doing so well. And I have no problem throwing out sociology and anthropology.

Don’t throw out the baby with the bathwater.
Post reply on HN