Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

statmodeling.stat.columbia.edu

Post reply on HN
Print view View original

111–120 of 243 posts

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#111

Post by evgen »

School-Cotton wrote: post #41
an_opabinia wrote: post #31

Andrew Gelman designed the 538 model in 2007. Nate Silver authored an adjustment to polls used in that model. Polls have more impact if they are more representative of statewide turnout among demographic things he chose like “black” and “low income.” This is why his predictions were so accurate for Obama’s 2008 and 2012 elections, and likely why they were so inaccurate in 2016. Gelman’s own grad student is the only p…

Nate Silver said Trump had a 1 in 3 chance, which basically means one shouldn’t be surprised no matter the result. I’m not sure where this “all the polls were so far off in 2016!!” narrative comes from, but it’s wrong.

And in particular any claim 538 was the site that was off the mark compared to other prediction sites is clearly based in a reality that is not shared with the rest of us. In the week before the election Nate and crew were posting articles specifically outlining the non-zero probability of a Trump win and if it happened how it was likely to happen.

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#112

Post by croissants »

justinzollars wrote: post #96
croissants wrote: post #88

Earlier quoted context omitted.

That's an interesting thought, but San Francisco aside, it seems like most people moving out of cities are just moving to the suburbs of those cities, which shouldn't affect the presidential election calculus?

I know people that have moved, but forgot to register. Its a fact of life that voter registration will take the back seat when managing more complicated things in life - like a move

That seems plausible. Still, I think the "exodus from cities" theory doesn't apply much to American cities outside of New York and San Francisco. Places like Philadelphia may have even seen a small uptick from NY emigration. But this is just a guess, it would be nice to see any statistics (I haven't found any).

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#113

Post by bonzini »

untog wrote: post #80
School-Cotton wrote: post #41

Earlier quoted context omitted.

Nate Silver said Trump had a 1 in 3 chance, which basically means one shouldn’t be surprised no matter the result. I’m not sure where this “all the polls were so far off in 2016!!” narrative comes from, but it’s wrong.

Nate was the outlier in that respect. But it’s true that the polls aren’t weren’t all that inaccurate in 2016: a bunch of important swing states were within the margin of error and Trump won some important states by very small margins. The mistake in 2016, IMO was a) the extrapolation that came from those polls and b) people paying way too much attention to national polls, which have very little connection to elector…

What Nate Silver got right in 2016 were the correlations in the Rust Belt, which were traditionally considered Democrat. 538's model predicted that losing one of those states would likely mean losing all of them for Clinton, because for example the polling errors were likely correlated. And indeed losing there is what cost her the election

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#114

Post by bigbubba »

unwoundmouse wrote: post #48
newfeatureok wrote: post #27

The thing about this election is that what if Trump himself is some sort of wildcard that can't really be properly forecasted in the polls? Why is there so much fascination with polls to begin with? I understand that there are betting markets, but it seems sort of silly. If you had a 100% accurate poll, for instance, then what would be the purpose of the actual election?

Now that you bring that up, if we had a 100% accurate poll that would be really good for productivity wouldn't it? Perhaps it wouldn't give voters the same feeling of self-determination but it'd save a lot of resources in fundraising, going out to vote, counting votes

The sensation of self determination is the entire point of democracy though. We don't use democracy because we think masses of people are particularly wise; we use these systems because they feel more fair than the alternatives and that perception of fairness produces good results (peaceful power transitions.)

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#115

Post by ta8645 »

cwhiz wrote: post #28

Not sure if it’s wrong to put this here, but here is a link to their election forecast. https://projects.economist.com/us-2020-forecast/president You can compare this to the 538 model and see where these two teams and forecasts disagree.

Can someone help me understand what odds like this mean in the context of an election?

The model says that Trump has a 1 in 10 chance of winning. With a fair 10-sided die it makes sense that you have a 1 in 10 chance of any given side rolling face up. But what is the die that is being rolled in these election statistics? What is the "chance" element that is being predicted?

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#116

Post by noirbot »

jimmygrapes wrote: post #61
ceilingcorner wrote: post #42

Earlier quoted context omitted.

> That's why this time around the pollsters made sure to be more thorough in their polling. "This time is different." I've heard that enough times to be highly skeptical. I'm also deeply skeptical of the notion that polling is even remotely correlated to actual results. Cultural and historical trends play a drastically higher role and are almost always left out.

The shy Tory factor[0] is likely to be even stronger this year than in 2016 when it comes to polls. After 4+ years of being harangued and called every name in the book by the vast majority of national culture (movies, music, TV, news, social media, news-entertainment), along with increasingly hostile projects such as https://donaldtrump.watch/ I imagine less of his more subdued supporters are going to be honest with…

But wouldn't that mostly be concentrated in places/areas where their votes aren't likely to matter? Having just driven through the US South in the last month, I can confidently tell you people are not in any way shy about their support for Trump. I saw more Trump signs and flags than I saw US flags.

It's also likely this works in both directions - if you support Biden, I bet you don't have a yard sign for it if you live in Mississippi.

By construction, the effect is strongest the more you're not in the majority, which also means your unspoken support is more likely to not matter on the actual outcomes of the election.

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#117

Post by thaumasiotes »

justinzollars wrote: post #44

Every single one of these models will break down this year. We are living in an unprecedented time. I can't understand how we can model how many people will vote, when we don't even know how many people have moved out of cities this year. Half of my friends have left San Francisco - if as many people left Philadelphia, The Twin cities, Milwaukee or Pittsburgh, then that really effects the outcome.

> I can't understand how we can model how many people will vote, when we don't even know how many people have moved out of cities this year. Half of my friends have left San Francisco

Moving out doesn't stop you from voting. I didn't change my voter registration when I moved from San Francisco to China. Years later, back in California, I voted in San Francisco, where I was still registered, despite residing in Hayward.

For verification purposes, they asked me when I voted what my address was. I was allowed to vote despite not knowing my own apartment number.

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#118

Post by wtallis »

ceilingcorner wrote: post #108
wtallis wrote: post #103

Earlier quoted context omitted.

You started off by calling the 2016 prediction a debacle , but now you're saying you would have put the odds about 12 percentage points differently. That doesn't seem like a big enough disagreement to warrant the kind of vehement criticism you're throwing around.

I called it a debacle because 99.9% of media sources, pundits, politicians, political figures, or anyone else thought that Hillary had anything less than a guaranteed win. I'm simply suggesting it was actually always a close race, but that the media ignored this because it went against their ideological model / they weren't familiar with places like the Rust Belt.

You really need to be more clear about which of your criticisms are against FiveThirtyEight specifically vs against the media in general. Because now it's looking like you are complaining about the media in general and using that as justification for mistrusting FiveThirtyEight, in the context of a discussion specifically about FiveThirtyEight being a notable outlier from that general media trend.

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#119

Post by ceilingcorner »

noirbot wrote: post #116
jimmygrapes wrote: post #61

Earlier quoted context omitted.

The shy Tory factor[0] is likely to be even stronger this year than in 2016 when it comes to polls. After 4+ years of being harangued and called every name in the book by the vast majority of national culture (movies, music, TV, news, social media, news-entertainment), along with increasingly hostile projects such as https://donaldtrump.watch/ I imagine less of his more subdued supporters are going to be honest with…

But wouldn't that mostly be concentrated in places/areas where their votes aren't likely to matter? Having just driven through the US South in the last month, I can confidently tell you people are not in any way shy about their support for Trump. I saw more Trump signs and flags than I saw US flags. It's also likely this works in both directions - if you support Biden, I bet you don't have a yard sign for it if you l…

It's more relevant in swing states, which are by definition mixed. I.e. if you live in Pennsylvania, Michigan, Ohio, or Florida, you can't really be sure how your neighbors will react to a Biden/Trump sign.

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#120

Post by an_opabinia »

thaumasiotes wrote: post #71
an_opabinia wrote: post #31

Andrew Gelman designed the 538 model in 2007. Nate Silver authored an adjustment to polls used in that model. Polls have more impact if they are more representative of statewide turnout among demographic things he chose like “black” and “low income.” This is why his predictions were so accurate for Obama’s 2008 and 2012 elections, and likely why they were so inaccurate in 2016. Gelman’s own grad student is the only p…

> Polls have more impact if they are more representative of statewide turnout among demographic things he chose like “black” and “low income.” This is why his predictions were so accurate for Obama’s 2008 and 2012 elections I find this argument strange, because black turnout was unusually high in 2008. That should have a negative impact on the accuracy of statistical adjustments, not a positive one.

I think he made an estimate for the increase in black turnout. If I were designing the model, and I believed turnout is the biggest factor (maybe inconclusive among political scientists), I would look at the circumstances where turnout changes based on candidate's demographics and validate it across statewide and congressional races.

However, we will never know, because they never published the code.

Post reply on HN