Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

statmodeling.stat.columbia.edu

Post reply on HN
Print view View original

91–100 of 243 posts

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#91

Post by ordinaryperson »

vanattab wrote: post #56
School-Cotton wrote: post #41

Earlier quoted context omitted.

Nate Silver said Trump had a 1 in 3 chance, which basically means one shouldn’t be surprised no matter the result. I’m not sure where this “all the polls were so far off in 2016!!” narrative comes from, but it’s wrong.

The narrative comes from the medias inaccurate and misleading coverage of the polls in 2016. Many news outlets all but declared Clinton president before the election.

But the media is not Nate Silver. He said Trump had about as much chance of winning as the Cubs had that year of wining the World Series, and obviously both happened.

Silver did a nice writeup of the whole experience: https://fivethirtyeight.com/features/the-real-story-of-2016/

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#92

Post by yongjik »

ceilingcorner wrote: post #53
delecti wrote: post #40

Earlier quoted context omitted.

It's not "wrong" to predict a low chance for something that eventually happens. Unlikely events can happen.

That isn't the criticism. The criticism is the appellation of it being "unlikely." For example: anyone paying attention to the Rust Belt ±1980-2016 would have dramatically upped Trump's chances in Pennsylvania and Michigan. FiveThirtyEight had Hillary with 70%+ chance of winning both, which to me, shows a deep ignorance of actual cultural factors.

> anyone paying attention to the Rust Belt ±1980-2016 would have dramatically upped Trump's chances in Pennsylvania and Michigan.

There was a very decent chance that Clinton could have won in 2016 (if any factor had gone slightly better for her), and if that had happened, nobody would be saying this now. This is literal hindsight bias.

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#93

Post by justinzollars »

ceilingcorner wrote: post #47
justinzollars wrote: post #44

Every single one of these models will break down this year. We are living in an unprecedented time. I can't understand how we can model how many people will vote, when we don't even know how many people have moved out of cities this year. Half of my friends have left San Francisco - if as many people left Philadelphia, The Twin cities, Milwaukee or Pittsburgh, then that really effects the outcome.

I've made similar points on topics like this, and bar none, every single time, it is downvoted into oblivion. People seem to have a difficult time with forecasting data that goes against their preferred outcomes.

I'm not trying to get points, or keep points. I'm just trying to point out that the models are not going to work this year and its crazy to assume they will.

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#94

Post by bart_spoon »

bena wrote: post #12
Marazan wrote: post #10

Earlier quoted context omitted.

If I tell you an unweighted 6 sided die only has a 1 in 6 chance of coming up 6 and we roll it once and it comes up a 6 then that is not junk science.

Yeah, but what if we could never roll that particular die again? I think that's what he's talking about.

Yes, which is why it’s foolish to talk about the outcome of that single race. You are right that there will never be another 2016 election between Clinton and Trump. However, that is only one of hundreds of forecasts made by 538 across multiple election cycles, so we can see how often their probabilistic outcomes align with actual outcomes. The fact that their track record is fairly good across all of these is evidence suggesting that Trump’s win may be more likely to be the realization of a less likely outcome predicted by their model, rather than a fundamental problem of their model itself.

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#95

Post by pjanoman »

ceilingcorner wrote: post #34
abnry wrote: post #33

Earlier quoted context omitted.

While numerically literate, I don't understand the details of the 538 or economist models. What I do know is that 538's model has had a great track record. It gave Trump one of the highest chances of winning in 2016. It did very well in prior elections. And both models are essentially predicting the same results: ~10% chance of Trump winning.

How is being slightly less wrong than everyone else "having a great track record"? Serious question. Because I find it hard to take any of them seriously after the debacle that was 2016.

At the end of the 2016 election, Donald Trump had a predicted 1/4 chance of winning the presidency. Does this seem like a massive debacle that Trump won under these conditions? Not to me.

https://projects.fivethirtyeight.com/2016-election-forecast/

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#96

Post by justinzollars »

croissants wrote: post #88
justinzollars wrote: post #44

Every single one of these models will break down this year. We are living in an unprecedented time. I can't understand how we can model how many people will vote, when we don't even know how many people have moved out of cities this year. Half of my friends have left San Francisco - if as many people left Philadelphia, The Twin cities, Milwaukee or Pittsburgh, then that really effects the outcome.

That's an interesting thought, but San Francisco aside, it seems like most people moving out of cities are just moving to the suburbs of those cities, which shouldn't affect the presidential election calculus?

I know people that have moved, but forgot to register. Its a fact of life that voter registration will take the back seat when managing more complicated things in life - like a move

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#97

Post by bigbubba »

cwhiz wrote: post #87
bigbubba wrote: post #39

Earlier quoted context omitted.

Some of your sibling comments seem to suggest otherwise.

Well, HN commenters don’t speak for the people doing election forecasting. I think Silver would describe it as educated guesswork.

I suppose you meant that election forecasters don't claim election forecasting is a science, not 'no one.' I don't mean to quibble, this meaning was not at all clear in your original comment, nor did I expect that meaning from the context of the comment you responded to.

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#98

Post by [deleted] »

Ibethewalrus wrote: post #64
hn_throwaway_99 wrote: post #24

Earlier quoted context omitted.

Are you referring to the 2016 election? If so, you are wrong. 538 gave Trump a higher chance of winning than pretty much every independent pollster.

nope, you're wrong. 538 gave Hillary a 71.4% chance of winning https://projects.fivethirtyeight.com/2016-election-forecast/

[deleted]

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#99

Post by noirbot »

Traster wrote: post #90

> I’d think that if Trump were to win New Jersey or, even more so, California, that this would most likely happen only as part of a national landslide of the sort envisioned by Scott Adams or whatever. That's a valid intuition to have but you can also clearly make the argument that if Trump wins California you're in such a weird scenario that using the traditional wisdom about correlation is dangerous. The point that…

It is worth noting that they explicitly aren't accounting for any election tampering/throwing out ballots in the model, so that final hypothetical isn't something they're factoring in. Your "an explosion kills everyone in LA" is closer to the sort of things that the model is "considering" in so much as a pile of statistical code has any understanding of what could happen in the real world to cause the outcome it's putting forward.

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#100

Post by ceilingcorner »

yongjik wrote: post #92
ceilingcorner wrote: post #53

Earlier quoted context omitted.

That isn't the criticism. The criticism is the appellation of it being "unlikely." For example: anyone paying attention to the Rust Belt ±1980-2016 would have dramatically upped Trump's chances in Pennsylvania and Michigan. FiveThirtyEight had Hillary with 70%+ chance of winning both, which to me, shows a deep ignorance of actual cultural factors.

> anyone paying attention to the Rust Belt ±1980-2016 would have dramatically upped Trump's chances in Pennsylvania and Michigan. There was a very decent chance that Clinton could have won in 2016 (if any factor had gone slightly better for her), and if that had happened, nobody would be saying this now. This is literal hindsight bias.

Aren't you doing the exact same thing that you're accusing me of?

My view is simple: the media completely, totally got 2016 wrong, mostly for sociological reasons. The people making the predictions simply had a huge blind spot. Brexit is another similar situation. The fact that Hillary almost won or Brexit almost didn't happen isn't really the point, because both things were never expected to be even remotely that close. Had the predictions been "Pennsylvania will be close", it would be relevant, but those weren't the predictions.

Post reply on HN