Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

statmodeling.stat.columbia.edu

Post reply on HN
Print view View original

151–160 of 243 posts

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#151

Post by FrojoS »

untog wrote: post #80
School-Cotton wrote: post #41

Earlier quoted context omitted.

Nate Silver said Trump had a 1 in 3 chance, which basically means one shouldn’t be surprised no matter the result. I’m not sure where this “all the polls were so far off in 2016!!” narrative comes from, but it’s wrong.

Nate was the outlier in that respect. But it’s true that the polls aren’t weren’t all that inaccurate in 2016: a bunch of important swing states were within the margin of error and Trump won some important states by very small margins. The mistake in 2016, IMO was a) the extrapolation that came from those polls and b) people paying way too much attention to national polls, which have very little connection to elector…

I think it’s a confusion between the likelihood of winning, no matter by how many votes, and the predicted percentage of votes per candidate. The latter is more commonly presented to readers from polls. So it’s not too surprising if it gets mixed up with the former, which is used by Nate et al and uses also percent as the unit.

Say a national poll predicts 55% of votes for Clinton, 40% for Trump. Whereas 538 predicts 70% chance of winning for Clinton and 30% for Trump. It’s easy to confuse the two and think the second prediction is much better for Clinton when it might be much worse.

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#152

Post by aecay »

weerd wrote: post #144
ta8645 wrote: post #115

Earlier quoted context omitted.

Can someone help me understand what odds like this mean in the context of an election? The model says that Trump has a 1 in 10 chance of winning. With a fair 10-sided die it makes sense that you have a 1 in 10 chance of any given side rolling face up. But what is the die that is being rolled in these election statistics? What is the "chance" element that is being predicted?

The odds for a face on a d10 would be 1 in 9 (1:9). It's different than probability (1/10 = .1)

It's typical to use "to" rather than "in" when discussing odds. So the odds of getting a 1 on a 10-sided dice are 9 to 1 against (odds are also typically specified with the larger number first, because of the overlap between mathematical odds and betting-shop odds). And the probability of it happening is 1 in 10.

(I suspect that counter-pedantry on these lines might be part of why your post is getting downvoted; I wasn't one of the downvoters fwiw.)

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#153

Post by yodon »

screye wrote: post #145

I disagree with the author on the idea that tail is too fat for isolated anomalies. There are most certainly events that can happen, which may lead to a red California or a blue Alabama. Presidential assassination, war, video proof of something incredibly heinous (pedophilia?), etc. can absolutely lead to these outcomes. You don't even have to go that far back. Nixon and Reagan flipped states like no-one's business.…

I've always liked Enrico Fermi's attitude on this. When you're Enrico Fermi, you get to say things like "One data point gives you a curve. Two data points gives you the distribution about the curve."

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#154

Post by fisherjeff »

I’ve decided to start my own election forecasting site that only ever gives 50/50 odds. Then I’ll just have to wait for the next 2016-style underdog win and my inevitable victory lap in the press as The Guy Who Called the Election.

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#155

Post by [deleted] »

RivieraKid wrote: post #105
nullc wrote: post #17

Meh. If you fit a model and don't explicitly constrain against "un-physical" results like negative correlations, you'll end up with them. Constraining against them won't improve your models fit (usually by definition), and it doesn't always improve robustness (at least for situations near average)-- because they're acting to debias the model in ways that you otherwise don't have enough degrees of freedom to address.…

> If you fit a model and don't explicitly constrain against "un-physical" results like negative correlations, you'll end up with them. The Economist model does exactly that, and all of their correlations are positive. I recommend reading their methodology, they know what they're doing (I wouldn't say the same about 538). Andrew Gelman has developed some of the Bayesian methods and software that people like Nate Silve…

[deleted]

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#156

Post by Balgair »

cwhiz wrote: post #28

Not sure if it’s wrong to put this here, but here is a link to their election forecast. https://projects.economist.com/us-2020-forecast/president You can compare this to the 538 model and see where these two teams and forecasts disagree.

Nit: Have they counted for the possibility of a tie? US elections allow for a tie in the Electoral College (which then kicks off a supremely strange and legalistic process).

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#157

Post by cwhiz »

Balgair wrote: post #156
cwhiz wrote: post #28

Not sure if it’s wrong to put this here, but here is a link to their election forecast. https://projects.economist.com/us-2020-forecast/president You can compare this to the 538 model and see where these two teams and forecasts disagree.

Nit: Have they counted for the possibility of a tie? US elections allow for a tie in the Electoral College (which then kicks off a supremely strange and legalistic process).

I can’t speak for all models but I do know that the 538 model accounts for a tie.

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#158

Post by Jabbles »

Balgair wrote: post #156
cwhiz wrote: post #28

Not sure if it’s wrong to put this here, but here is a link to their election forecast. https://projects.economist.com/us-2020-forecast/president You can compare this to the 538 model and see where these two teams and forecasts disagree.

Nit: Have they counted for the possibility of a tie? US elections allow for a tie in the Electoral College (which then kicks off a supremely strange and legalistic process).

Before commenting you could at least search for the word "tie":

The probability of an electoral-college tie is

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#159

Post by albntomat0 »

ta8645 wrote: post #115
cwhiz wrote: post #28

Not sure if it’s wrong to put this here, but here is a link to their election forecast. https://projects.economist.com/us-2020-forecast/president You can compare this to the 538 model and see where these two teams and forecasts disagree.

Can someone help me understand what odds like this mean in the context of an election? The model says that Trump has a 1 in 10 chance of winning. With a fair 10-sided die it makes sense that you have a 1 in 10 chance of any given side rolling face up. But what is the die that is being rolled in these election statistics? What is the "chance" element that is being predicted?

This sounds like a frequentist vs Bayesian statistics discussion, which involves (this is a simplification by me, a non-expert in the area) different definitions of probability. The frequentist view is along the lines of rolling a 10 side die hundreds of times, recording the results, and determining that each side comes up equally. The Bayesian view is that the probability measures our certainty about some event. For example, take the hypothetical point in time where all ballots have been cast, but have not been counted. One could use polling data, etc to model the odds that a particular candidate has won. However, the frequentist approach doesn’t really make sense here, as the ground truth already exists (all ballots cast), so rerunning the the event doesn’t make sense.

Once again, I’m not an expert, so I recommend looking for additional explanations, if you’re interested.

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#160

Post by weerd »

aecay wrote: post #152
weerd wrote: post #144

Earlier quoted context omitted.

The odds for a face on a d10 would be 1 in 9 (1:9). It's different than probability (1/10 = .1)

It's typical to use "to" rather than "in" when discussing odds. So the odds of getting a 1 on a 10-sided dice are 9 to 1 against (odds are also typically specified with the larger number first, because of the overlap between mathematical odds and betting-shop odds). And the probability of it happening is 1 in 10. (I suspect that counter-pedantry on these lines might be part of why your post is getting downvoted; I wa…

TIL, thank you. Looking back, not a great post. Probably deserves the votes.
Post reply on HN