Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

statmodeling.stat.columbia.edu

Post reply on HN
Print view View original

31–40 of 243 posts

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#31

Post by an_opabinia »

Andrew Gelman designed the 538 model in 2007.

Nate Silver authored an adjustment to polls used in that model. Polls have more impact if they are more representative of statewide turnout among demographic things he chose like “black” and “low income.” This is why his predictions were so accurate for Obama’s 2008 and 2012 elections, and likely why they were so inaccurate in 2016.

Gelman’s own grad student is the only person to have academically published this approach, in a paper about polling Xbox Live users.

These guys sort of make a thing that is the same in many more ways than it is different. Why not just share the code is the biggest question?

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#32

Post by ceilingcorner »

newfeatureok wrote: post #27

The thing about this election is that what if Trump himself is some sort of wildcard that can't really be properly forecasted in the polls? Why is there so much fascination with polls to begin with? I understand that there are betting markets, but it seems sort of silly. If you had a 100% accurate poll, for instance, then what would be the purpose of the actual election?

The soothsayers and fortune tellers never went out of business, they just adopted a new name.

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#33

Post by abnry »

RivieraKid wrote: post #25

I'm surprised that a lot of people almost worship the 538 model, when there is a better model with more competent people behind it. Economist is also more open about how their model works and it's open-source. After reading the methodology I was pretty impressed. Edit: Why am I being downvoted?

While numerically literate, I don't understand the details of the 538 or economist models. What I do know is that 538's model has had a great track record. It gave Trump one of the highest chances of winning in 2016. It did very well in prior elections. And both models are essentially predicting the same results: ~10% chance of Trump winning.

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#34

Post by ceilingcorner »

abnry wrote: post #33
RivieraKid wrote: post #25

I'm surprised that a lot of people almost worship the 538 model, when there is a better model with more competent people behind it. Economist is also more open about how their model works and it's open-source. After reading the methodology I was pretty impressed. Edit: Why am I being downvoted?

While numerically literate, I don't understand the details of the 538 or economist models. What I do know is that 538's model has had a great track record. It gave Trump one of the highest chances of winning in 2016. It did very well in prior elections. And both models are essentially predicting the same results: ~10% chance of Trump winning.

How is being slightly less wrong than everyone else "having a great track record"? Serious question. Because I find it hard to take any of them seriously after the debacle that was 2016.

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#35

Post by abnry »

ceilingcorner wrote: post #34
abnry wrote: post #33

Earlier quoted context omitted.

While numerically literate, I don't understand the details of the 538 or economist models. What I do know is that 538's model has had a great track record. It gave Trump one of the highest chances of winning in 2016. It did very well in prior elections. And both models are essentially predicting the same results: ~10% chance of Trump winning.

How is being slightly less wrong than everyone else "having a great track record"? Serious question. Because I find it hard to take any of them seriously after the debacle that was 2016.

They said Trump had a 1 in 4 chance. That's very high. NYT had something like 1 in 20 chance for Trump.

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#36

Post by ceilingcorner »

abnry wrote: post #35
ceilingcorner wrote: post #34

Earlier quoted context omitted.

How is being slightly less wrong than everyone else "having a great track record"? Serious question. Because I find it hard to take any of them seriously after the debacle that was 2016.

They said Trump had a 1 in 4 chance. That's very high. NYT had something like 1 in 20 chance for Trump.

That doesn't really answer my question. It only indicates that they were slightly less wrong than every other media source, not that they have a good model.

If I had a laptop that only worked 1/4th of the time, rather than 1/20th of the time, would that make it a reliable laptop? I don't think so.

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#37

Post by techsupporter »

newfeatureok wrote: post #27

The thing about this election is that what if Trump himself is some sort of wildcard that can't really be properly forecasted in the polls? Why is there so much fascination with polls to begin with? I understand that there are betting markets, but it seems sort of silly. If you had a 100% accurate poll, for instance, then what would be the purpose of the actual election?

> Why is there so much fascination with polls to begin with?

Polling is useful for candidates and gives them ideas on where to target outreach and spending.

For the rest of us, it gives us something to watch. With Presidential campaigns running for almost two years in advance of the actual voting weeks, there’s a huge gap between when the thing starts and when we see results. This way, people have something to fill the time. Even now that voting has started, we are still another 10 days until the voting is done and likely another 7 after that until we have a sufficient count to know who has been elected.

That’s a long time for a populace that’s worried, distracted, and interested, especially since so many of us live in states where—due to the mechanics of a broken election system—we can’t do much to influence the national outcome.

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#38

Post by gamblor956 »

ceilingcorner wrote: post #34
abnry wrote: post #33

Earlier quoted context omitted.

While numerically literate, I don't understand the details of the 538 or economist models. What I do know is that 538's model has had a great track record. It gave Trump one of the highest chances of winning in 2016. It did very well in prior elections. And both models are essentially predicting the same results: ~10% chance of Trump winning.

How is being slightly less wrong than everyone else "having a great track record"? Serious question. Because I find it hard to take any of them seriously after the debacle that was 2016.

They were generally correct with their prediction except in 3 states where nobody had been doing detailed polling because the pollsters didn't think it would matter.

It simply wouldn't have been possible for the models to be more accurate with the data they had. As they say, bad data in, bad data out.

That's why this time around the pollsters made sure to be more thorough in their polling.

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#39

Post by bigbubba »

cwhiz wrote: post #15
KKKKkkkk1 wrote: post #6

My biggest issue is when people say that it's a probabilistic model, and therefore it wasn't wrong in 2016 because 28% chance of winning is pretty high and you don't get probabilities. Well, guess what, this kind of model that provides a probabilistic estimate on a future event that cannot be repeated cannot be validated or falsified. It's basically junk science (if it has any aspirations of being scientific).

No one is purporting election forecasting to be scientific.

Some of your sibling comments seem to suggest otherwise.

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#40

Post by delecti »

ceilingcorner wrote: post #36
abnry wrote: post #35

Earlier quoted context omitted.

They said Trump had a 1 in 4 chance. That's very high. NYT had something like 1 in 20 chance for Trump.

That doesn't really answer my question. It only indicates that they were slightly less wrong than every other media source, not that they have a good model. If I had a laptop that only worked 1/4th of the time, rather than 1/20th of the time, would that make it a reliable laptop? I don't think so.

It's not "wrong" to predict a low chance for something that eventually happens. Unlikely events can happen.
Post reply on HN