Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Put not thy trust in Nate Silver

thenewatlantis.com

Post reply on HN
Print view View original

11–20 of 26 posts

Re: Put not thy trust in Nate Silver

#11

Post by bleah1000 »

The article is way too long, and doesn't really talk about Nate Silver much.

However, one interesting item is the idea that there is no way to verify these predictions. Because they are forecasting one time events, there is no way to validate if their models are in the least bit accurate. If you have a X% chance of event happening and it doesn't, were you wrong? No because it wasn't 100%, and even if you are right, it's not terribly meaningful. It would be interesting to see if a site like five thirty-eight modeled all elections results, and then see how accurate they were across all of their predictions. Maybe you would have to look at all predictions with > 70% chance of happening and see if they got 70% of those correct.

However, in the end those predictions seem like they could be done by throwing darts at a dart board and we wouldn't really know.

Re: Put not thy trust in Nate Silver

#12

Post by scarmig »

bleah1000 wrote: post #11

The article is way too long, and doesn't really talk about Nate Silver much. However, one interesting item is the idea that there is no way to verify these predictions. Because they are forecasting one time events, there is no way to validate if their models are in the least bit accurate. If you have a X% chance of event happening and it doesn't, were you wrong? No because it wasn't 100%, and even if you are right, i…

It's not as dire as you suggest. To simplify a bit, you can take the set of ~75% predictions created by a model (which can be an actual model, or it can be just the meta model of whatever model Nate Silver wants us to trust right now), and then see what percentage of them were right. If that comes to 75%, then the model is well-calibrated, and it's a solid enough basis to assign a 75% probability to future 75% predictions, which is often as good as you can get when trying to make decisions and plans.

Re: Put not thy trust in Nate Silver

#13

Post by borramakot »

bleah1000 wrote: post #11

The article is way too long, and doesn't really talk about Nate Silver much. However, one interesting item is the idea that there is no way to verify these predictions. Because they are forecasting one time events, there is no way to validate if their models are in the least bit accurate. If you have a X% chance of event happening and it doesn't, were you wrong? No because it wasn't 100%, and even if you are right, i…

Boy, have I got the incredibly specific article for that exact question: https://fivethirtyeight.com/features/when-we-say-70-percent-...

Re: Put not thy trust in Nate Silver

#14

Post by mabbo »

bleah1000 wrote: post #11

The article is way too long, and doesn't really talk about Nate Silver much. However, one interesting item is the idea that there is no way to verify these predictions. Because they are forecasting one time events, there is no way to validate if their models are in the least bit accurate. If you have a X% chance of event happening and it doesn't, were you wrong? No because it wasn't 100%, and even if you are right, i…

> those predictions seem like they could be done by throwing darts at a dart board

They are. They're made by throwing 40,000 darts at a dart board. 40,000 simulations of the election based on the polls, historical trends, demographics, and more.

The real threat, the real reason why the 2016 election was mostly mis-called, is when the polls themselves make mistakes in a common way. In 2016 it was education that wasn't factored strongly enough.

The models can be run on past elections and polling + data before those elections. That's how the develop the models.

Re: Put not thy trust in Nate Silver

#15

Post by borramakot »

borramakot wrote: post #13
bleah1000 wrote: post #11

The article is way too long, and doesn't really talk about Nate Silver much. However, one interesting item is the idea that there is no way to verify these predictions. Because they are forecasting one time events, there is no way to validate if their models are in the least bit accurate. If you have a X% chance of event happening and it doesn't, were you wrong? No because it wasn't 100%, and even if you are right, i…

Boy, have I got the incredibly specific article for that exact question: https://fivethirtyeight.com/features/when-we-say-70-percent-...

I don't think this article provides strong evidence that they are well calibrated on the presidential election specifically (sample size N=3), or that they are correctly accounting for rare black swan events, but it does seem to imply that the criticisms about "538 claims victory no matter what because they always have non-zero probabilities" are oversimplified.

Re: Put not thy trust in Nate Silver

#16

Post by [deleted] »

scarmig wrote: post #6

It's pretty lame to paste an image of the NY Time's 82% chance of Clinton winning the election in an article that calls out Nate by name. Nate gave Clinton like a 70% chance. Can model building predict human behavior? It's fuzzier than the hard sciences, but not that much fuzzier. And if you do believe it's impossible to model human behavior, you have to throw out vast swaths of knowledge and areas of study: there go…

[deleted]

Re: Put not thy trust in Nate Silver

#17

Post by smitty1e »

xhkkffbf wrote: post #2

I think Nate is pretty sharp -- but I don't trust the people who answer polls. Who has time to pick up the phone? Most of the time it's some spammer. And if you do answer, why tell the truth about your vote?

Trump in 2016, and again in 2020 is running a disintermediation campaign and going straight to the voters with the rallies.

If again effective, perhaps the teachers will return to school and learn something.

Re: Put not thy trust in Nate Silver

#20

Post by bassman9000 »

mabbo wrote: post #8

Nate Silver and 538 gave Trump a 28% chance of winning on Election night in 2016. Nobody else gave him even close to that kind of probability. Sam Wang, considered another very good polling expert said he would eat a bug if Trump won, giving him less than 1% of a chance (and he DID[0]). And here is the New Atlantic saying not to trust Nate Silver? Give me a break. Statistical models give probabilities, not certaintie…

https://projects.fivethirtyeight.com/2020-election-forecast/...

89% chance to Biden. We'll see in a few days.

Post reply on HN