Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

statmodeling.stat.columbia.edu

Post reply on HN
Print view View original

211–220 of 243 posts

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#211

Post by perryizgr8 »

rsynnott wrote: post #191
untog wrote: post #80

Earlier quoted context omitted.

Nate was the outlier in that respect. But it’s true that the polls aren’t weren’t all that inaccurate in 2016: a bunch of important swing states were within the margin of error and Trump won some important states by very small margins. The mistake in 2016, IMO was a) the extrapolation that came from those polls and b) people paying way too much attention to national polls, which have very little connection to elector…

> But people read “90% chance” as “definite win”. I don’t actually know what anyone should or could do about that. 538 are aware of the problem, and combating it with a cartoon fox (and better visualisations).

I don't know if this statement is serious. How does a cartoon fox make me think differently about the numbers?

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#212

Post by solveit »

lordnacho wrote: post #146
School-Cotton wrote: post #41

Earlier quoted context omitted.

Nate Silver said Trump had a 1 in 3 chance, which basically means one shouldn’t be surprised no matter the result. I’m not sure where this “all the polls were so far off in 2016!!” narrative comes from, but it’s wrong.

It comes from innumerate journalism, and an innumerate population. Next time someone laughs off being bad at math, you should point out that being unable to read is no laughing matter, and being unable to understand numbers shouldn't be either. The only sensible way to predict probabilities that aren't extreme is to tell people how the model works and the figures it is currently spitting out. That's is the great thin…

> Next time someone laughs off being bad at math, you should point out that being unable to read is no laughing matter, and being unable to understand numbers shouldn't be either.

You're not wrong, but you should not do this.

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#213

Post by swader999 »

StreamBright wrote: post #164
ta8645 wrote: post #115

Earlier quoted context omitted.

Can someone help me understand what odds like this mean in the context of an election? The model says that Trump has a 1 in 10 chance of winning. With a fair 10-sided die it makes sense that you have a 1 in 10 chance of any given side rolling face up. But what is the die that is being rolled in these election statistics? What is the "chance" element that is being predicted?

It pretty much means nothing. These sort of models produced wrong result again and again. For me the biggest question mark if that if we know that recent (last 5) elections were very close how can you predict somebody winning with 93% chance? Maybe I do not understand something here.

Agree. Polls are a bad metric to rely on. Only 2% of those asked respond. Impossible to get a statistical sample with that. People are bad at predicting their future behavior. They are dishonest, they don't know or just don't want to tell you. There's a huge class divide right now. The bigger the class divide the worse polls are historically. And we know they are wrong this time around. None of the early voting margins predicted have held close. Better predictors are Google trends, rally participation, and voter registrations.

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#214

Post by jerkstate »

gamblor956 wrote: post #135
ceilingcorner wrote: post #74

Earlier quoted context omitted.

Polarization and the unacceptability of publicly saying "I voted for X" also didn't really exist prior to 2016. The fear of getting doxxed, combined with a record low level of trust in institutions and the media, leads to skepticism toward answering polls truthfully, IMO.

While polarization is bad now, it's nowhere close to historical extremes, and it's not even as bad as it was during the post-Vietnam era only a few decades ago. As for fear of doxing: plenty of people openly supported and voted for George Wallace (a noted white Supremacist) back in the day, and even Roy Moore (accused pedophile) just 2 years ago. Proud Boys members openly pose for the cameras even as they espouse rac…

As a data point, I live in a conservative-leaning area of a purple/blue state. Along my normal driving routes I had seen quite a few, but over the past few weeks they have mostly been taken down (in every case, other republican candidate signs still stand). Scenario one, Trump supporters supported him all the way up until now, enough to donate to his re-election campaign to buy a yard sign, and in the final weeks of the election, decided that they'd had enough and taken down their signs. Scenario two, some random stranger decided that it's their right, nay, their responsibility, to moderate someone else's yard signs.

Not every Trump voter is a "Proud Boy" or noted white supremacist, maybe even some of your friends are who "just aren't interested in politics" because they know that if they were honest with you, you would flip out. Some understand that this attitude about yard signs is actually representative of an entire worldview, and opposing that might actually be the lesser of two evils.

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#215

Post by dllthomas »

dsaavy wrote: post #178
an_opabinia wrote: post #31

Andrew Gelman designed the 538 model in 2007. Nate Silver authored an adjustment to polls used in that model. Polls have more impact if they are more representative of statewide turnout among demographic things he chose like “black” and “low income.” This is why his predictions were so accurate for Obama’s 2008 and 2012 elections, and likely why they were so inaccurate in 2016. Gelman’s own grad student is the only p…

I think polls being more representative of turnout amongst minorities could help indicate a potential black swan event for the election. If turnout does return to 2008 and 2012 election levels, polls featured in this fivethirtyeight article [1] indicate Trump is performing better amongst black and hispanic voters. Both demographics are seeing a 10-15% swing in support for Trump compared to 2016, which could theoretic…

You're misreading or confused or something. The 538 piece points to a ~10% swing away from the (in the case of black voters) 82% that favored Hillary. This is very different from a swing all the way to a 10% preference for Trump. A bigger turnout by (these) minority voters, assuming they cast votes even vaguely in line with this polling, is more votes in Biden's column than Trump's. That's bad news for Trump.

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#216

Post by tomjakubowski »

perryizgr8 wrote: post #211
rsynnott wrote: post #191

Earlier quoted context omitted.

> But people read “90% chance” as “definite win”. I don’t actually know what anyone should or could do about that. 538 are aware of the problem, and combating it with a cartoon fox (and better visualisations).

I don't know if this statement is serious. How does a cartoon fox make me think differently about the numbers?

Like the coyote in El Viaje Misterioso de Nuestro Jomer, the cartoon fox tells you only just enough to light the path towards statistical enlightenment. You must walk it yourself.

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#217

Post by TTPrograms »

runawaybottle wrote: post #207
TTPrograms wrote: post #185

Earlier quoted context omitted.

To you and the other comment - as much as I dislike Taleb's rhetoric, this is precisely the sort of bet he has made a lot of money on. People round rare event probability down to zero, and if you bet against them enough with sufficiently extreme odds you'll eventually (and in expectation) hit a home run. I would be more than happy to make this bet with anyone willing to take the other side - as in literally, find a m…

But would Taleb specifically make a one time bet on one particular Black swan? Isn’t the idea the same as venture capitalism, where this idea only works if you do it with all possible black swans? You’d have to be crazy to take this one specific bet, you can only realistically take all possible improbable bets.

Choosing to engage in these sorts of bets systematically exclusively and not individually only makes sense if you're not maximizing expected return. You also don't know if I engage in these sorts of bets systematically.

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#218

Post by amscanne »

mrtnmcc wrote: post #110
paulgb wrote: post #29

Earlier quoted context omitted.

From the plots Andrew posted, it looks like the problem is not just sample size and that (some) individual state pairs have inverse correlations, e.g. https://statmodeling.stat.columbia.edu/wp-content/uploads/20...

I'd argue negative correlation on conditionals distributions can be reasonable here. In that particular WA-MS example, if Trump suddenly took more liberal positions and somehow won WA (e.g., announces he's pro abortion), he would in fact be more at risk of losing Mississippi. The idea that these two states are in play already is fringe and would require some major idealogical (or other third variable) shifts.

But then the correlations with the other 48 states are broken. In that insane scenario, Mississippi now votes for Biden (because I guess he’s suddenly come out as pro-life as well), but Alaska still goes for Trump.

The negative correlations don’t make sense. Maybe it’s a small problem and the model is solid overall, but... I don’t think you can justify that one effect.

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#219

Post by sangnoir »

RivieraKid wrote: post #105
nullc wrote: post #17

Meh. If you fit a model and don't explicitly constrain against "un-physical" results like negative correlations, you'll end up with them. Constraining against them won't improve your models fit (usually by definition), and it doesn't always improve robustness (at least for situations near average)-- because they're acting to debias the model in ways that you otherwise don't have enough degrees of freedom to address.…

> If you fit a model and don't explicitly constrain against "un-physical" results like negative correlations, you'll end up with them. The Economist model does exactly that, and all of their correlations are positive. I recommend reading their methodology, they know what they're doing (I wouldn't say the same about 538). Andrew Gelman has developed some of the Bayesian methods and software that people like Nate Silve…

My statistics knowledge has withered away, but isn't this quibbling over overall approach? 538 seems do be doing a top-down approach while the Economist is more bottom-up. What is strange is that a person affiliated to the Economist is then asking why the 538 model's emergent properties aren't exhibiting more bottom-up characteristics .

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#220

Post by sangnoir »

taeric wrote: post #205
i_love_limes wrote: post #177

Earlier quoted context omitted.

I think a lot of people (myself included when I've felt the pressure to) lowball how long things like this take because a) it makes me look smart and b) people could judge of they knew who much time I actually wasted on it.

I think you read the parent in the opposite direction than intended. It was praise for getting this done so quickly, by my read. Granted, i could just be misreading this post. :)

I read it as an encouragement for parent to not take the "hour" mentioned by TFA at face value, so they shouldn't be too hard on themselves.
Post reply on HN