The first problem with the model is it grossly missed the last election. I still think it’s good reading.
Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast
21–30 of 243 posts
Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast
#22See also this Twitter thread from Nate Cohn (who has no dog in the fight): https://twitter.com/Nate_Cohn/status/1320042092694065153
That's a pretty bold claim to make about essentially anybody in regards to the US presidential election. Not that I don't believe his account is even-handed and valuable, just that I'm curious what makes you say he has no dog in the fight.
Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast
#23FiveThirtyEight has a page where you can choose winning states (condition on a certain outcome) and it will regenerate the prediction map, https://projects.fivethirtyeight.com/trump-biden-election-ma... This appears to be the what Andrew Gelman is also trying to do with their raw data.
At the bottom of the 538 page it says, " If you choose enough unlikely outcomes, we’ll eventually wind up with so few simulations remaining that we can’t produce accurate results. When that happens, we go back to our full set of simulations and run a series of regressions to see how your scenario might look if it turned up more often."
I interpret that as running a regression (linear?) and extrapolating it out to the tail where the conditioning is happening. This should eliminate the issue Andrew is seeing?
Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast
#24The first problem with the model is it grossly missed the last election. I still think it’s good reading.
Are you referring to the 2016 election? If so, you are wrong. 538 gave Trump a higher chance of winning than pretty much every independent pollster.
Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast
#25I'm surprised that a lot of people almost worship the 538 model, when there is a better model with more competent people behind it. Economist is also more open about how their model works and it's open-source. After reading the methodology I was pretty impressed.
Edit: Why am I being downvoted?
Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast
#26My biggest issue is when people say that it's a probabilistic model, and therefore it wasn't wrong in 2016 because 28% chance of winning is pretty high and you don't get probabilities. Well, guess what, this kind of model that provides a probabilistic estimate on a future event that cannot be repeated cannot be validated or falsified. It's basically junk science (if it has any aspirations of being scientific).
If I tell you an unweighted 6 sided die only has a 1 in 6 chance of coming up 6 and we roll it once and it comes up a 6 then that is not junk science.
Sounds more like gambling than science; you're making a one in six bet that you can dupe me. More seriously, I would not accept your claim with evidence as weak as that. Roll the dice a few more times and then I'll credit your claim. Rolling the dice only once when you could just as easily roll them a dozen times is junk science.
Incidentally, this trick is something magicians sometimes do. Sometimes when a trick has gone wrong they'll make a wild guess. If they're right, the audience is impressed. If they're wrong, they'll brush it aside with some joke and the audience won't notice/mind much. This works for things card guessing tricks and puedo-psychic/cold reading stuff.
Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast
#27The thing about this election is that what if Trump himself is some sort of wildcard that can't really be properly forecasted in the polls?
Why is there so much fascination with polls to begin with? I understand that there are betting markets, but it seems sort of silly. If you had a 100% accurate poll, for instance, then what would be the purpose of the actual election?
Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast
#28Not sure if it’s wrong to put this here, but here is a link to their election forecast.
https://projects.economist.com/us-2020-forecast/president
You can compare this to the 538 model and see where these two teams and forecasts disagree.
Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast
#29FiveThirtyEight has a page where you can choose winning states (condition on a certain outcome) and it will regenerate the prediction map, https://projects.fivethirtyeight.com/trump-biden-election-ma... This appears to be the what Andrew Gelman is also trying to do with their raw data. At the bottom of the 538 page it says, " If you choose enough unlikely outcomes, we’ll eventually wind up with so few simulations rem…
From the plots Andrew posted, it looks like the problem is not just sample size and that (some) individual state pairs have inverse correlations, e.g. https://statmodeling.stat.columbia.edu/wp-content/uploads/20...
Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast
#30See also this Twitter thread from Nate Cohn (who has no dog in the fight): https://twitter.com/Nate_Cohn/status/1320042092694065153
That's a pretty bold claim to make about essentially anybody in regards to the US presidential election. Not that I don't believe his account is even-handed and valuable, just that I'm curious what makes you say he has no dog in the fight.
I think he is not talking about the election in general but referencing the "fight" between the 538 model/Nate Silver vs. the Economist/Andre Gelman.