Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast • Hacker News Board

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#181

Post by gridlockd » Sun, Oct 25, 2020, 10:28 PM UTC

sequoia wrote: post #175

As a US voter I am very frustrated With and tired of this obsession with election forecasting. Is it going to influence whether or not you go out and vote? If not, what is the point of it? What value does something like fivethirtyeight add to our democracy, if any? Is this motivation the same as that of diving deep into baseball stats or Star Wars starship engineering, just like “nerding out” for its own sake? Contra…

The point is that people want to know what's going to happen before it happens. If I believed a Democrat was sure to win, I would want to allocate my stock portfolio different than if a Republican were to win.

Also, you can make money betting on the outcome itself. If the odds you get are underpriced relative to an accurate forecast, that's a great bet to take.

Furthermore, these forecasts influence where politicians put their focus. Let's say you're Hillary in '16 and you think Wisconsin is yours despite the forecast showing a narrow lead, maybe you should reconsider.

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#182

Post by l8rpeace » Sun, Oct 25, 2020, 10:37 PM UTC

sequoia wrote: post #175

As a US voter I am very frustrated With and tired of this obsession with election forecasting. Is it going to influence whether or not you go out and vote? If not, what is the point of it? What value does something like fivethirtyeight add to our democracy, if any? Is this motivation the same as that of diving deep into baseball stats or Star Wars starship engineering, just like “nerding out” for its own sake? Contra…

Diving deep into baseball statistics has actually changed how the game is played. I understand your sentiment overall, I don't agree with your sentiment because there are valid use cases for predictive analysis in polotics, but regardless G whether I agree or disagree about political polling - using baseball analytics as an example is not appropriate.

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#183

Post by rayiner » Sun, Oct 25, 2020, 10:37 PM UTC

sequoia wrote: post #175

As a US voter I am very frustrated With and tired of this obsession with election forecasting. Is it going to influence whether or not you go out and vote? If not, what is the point of it? What value does something like fivethirtyeight add to our democracy, if any? Is this motivation the same as that of diving deep into baseball stats or Star Wars starship engineering, just like “nerding out” for its own sake? Contra…

It's worth pointing out that the other thing 538 does besides political polling is analyzing the performance of sports teams. Americans are just drawn to the horse race. (See also: American Idol, which also involves voting.)

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#184

Post by FinanceAnon » Sun, Oct 25, 2020, 10:38 PM UTC

sequoia wrote: post #175

As a US voter I am very frustrated With and tired of this obsession with election forecasting. Is it going to influence whether or not you go out and vote? If not, what is the point of it? What value does something like fivethirtyeight add to our democracy, if any? Is this motivation the same as that of diving deep into baseball stats or Star Wars starship engineering, just like “nerding out” for its own sake? Contra…

Apart from what others have said, I think it also good to have independent polls to shine light on fraudulent elections. For example, see what happened in Belarus recently where public polling is banned.

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#185

Post by TTPrograms » Sun, Oct 25, 2020, 10:45 PM UTC

artichokes wrote: post #172

TTPrograms wrote: post #147

For a post on a blog presumably focused on statistics the author seems oddly concerned with a model that predicts odd events as being "possible". The difference between "possible" and "impossible" is \eps Likewise asserting that California with a 3% chance of going Trump is absurd is an unreasonable degree of overconfidence. Assuming maximizing expected return, the author is implying that they would be willing to tak…

I don't know if I'm reading your comment wrong but I would happily take that bet- I think most people would. Want to make it? The election result of California isn't a matter of probability; it's an empirical matter that the number of people who will vote for Biden in California far exceeds the number that will vote for Trump.

To you and the other comment - as much as I dislike Taleb's rhetoric, this is precisely the sort of bet he has made a lot of money on. People round rare event probability down to zero, and if you bet against them enough with sufficiently extreme odds you'll eventually (and in expectation) hit a home run.

I would be more than happy to make this bet with anyone willing to take the other side - as in literally, find a modern middle-man system and I'm game.

You may want to be aware that you have provided me a nontrivial arbitrage opportunity, as the odds on predictit are closer to 7:93 : https://www.predictit.org/markets/detail/6611

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#186

Post by eyelidlessness » Sun, Oct 25, 2020, 10:48 PM UTC

sequoia wrote: post #175

As a US voter I am very frustrated With and tired of this obsession with election forecasting. Is it going to influence whether or not you go out and vote? If not, what is the point of it? What value does something like fivethirtyeight add to our democracy, if any? Is this motivation the same as that of diving deep into baseball stats or Star Wars starship engineering, just like “nerding out” for its own sake? Contra…

Well, I’m not particularly affected by this (I live in a state that hasn’t been considered purple for at least a decade and probably shouldn’t have been for the decade prior), but... most election systems have some form of strategic voting behavior—either built in (all manner of ranked voting), or based on understanding of the system and being underserved by it. In the US, that could look like leftish people in VA seeing that the state is reasonably safe to do its part to oust trump, and people of the same lean in NC not seeing the same level of safety. Or rightish people in Georgia feeling less safe bucking the Rs, and so on.

In this system, there are two ways to democratically influence politics by voting:

1. Vote your preferred candidate

2. Withhold your vote from the party more closely aligned with your views, in hopes of helping shift its coalition priorities

If you fall into the second category, accurate forecasting makes a strategic difference.

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#187

Post by runawaybottle » Sun, Oct 25, 2020, 10:59 PM UTC

aakilfernandes wrote: post #148

ta8645 wrote: post #115

Earlier quoted context omitted.

Can someone help me understand what odds like this mean in the context of an election? The model says that Trump has a 1 in 10 chance of winning. With a fair 10-sided die it makes sense that you have a 1 in 10 chance of any given side rolling face up. But what is the die that is being rolled in these election statistics? What is the "chance" element that is being predicted?

You're comparing two scenarios, one in which you know all the facts, and one in which you don't. In the dice toss scenario, we know everything relevant. In the election scenario, we don't. A model like this is attempting to say "these are the rules we think exist. Based on the rules, and assuming the data is off by some random distribution, here's what we think could happen". What different forecasters disagree about…

I will veer this off into the dreaded political territory even though this is mostly a technical discussion.

The Democratic Party proved it was not as progressive as they thought as Sanders lost the primary. The reality is, the country as a whole is also not as liberal either, regardless of what these pollsters are asking people. You think the party is youthful, and ready for progressive ideas, but alas, the party wholly rejected an amazingly progressive candidate in Sanders. You think everyone’s super pissed at Coronavirus handling, and police brutality, healthcare, but alas, you find out people associate BLM protests with crime, and the virus with China, and socialism with unfair wealth redistribution. We can keep learning this the hard way I guess, this is America after all.

It’s important the technical discussions are happening this time around, because there was virtually none the last time. The post mortems for these forecasts being wrong again should be a death knell for accumulating bad data. I’m certain the models are good, but I’m not certain the data is.

Anyway, if you want my hot take, the conditional forecasting is to save their ass on election night from being embarrassingly wrong again. Imagine writing a giant if-statement that looked something like ‘and if(imWrong) changeMyAnswer’.

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#188

Post by JJzD » Sun, Oct 25, 2020, 11:01 PM UTC

jimmygrapes wrote: post #61

ceilingcorner wrote: post #42

Earlier quoted context omitted.

> That's why this time around the pollsters made sure to be more thorough in their polling. "This time is different." I've heard that enough times to be highly skeptical. I'm also deeply skeptical of the notion that polling is even remotely correlated to actual results. Cultural and historical trends play a drastically higher role and are almost always left out.

The shy Tory factor[0] is likely to be even stronger this year than in 2016 when it comes to polls. After 4+ years of being harangued and called every name in the book by the vast majority of national culture (movies, music, TV, news, social media, news-entertainment), along with increasingly hostile projects such as https://donaldtrump.watch/ I imagine less of his more subdued supporters are going to be honest with…

If you're right, the support for Trump should be higher in polls where people talk through a phone menu or the internet, compared to live interviews.

Newsflash, they aren't, so your hypothesis must be they're too shy to admit their Trump support to a machine?

Everybody is just fighting the last war where Trump suddenly won for a couple of reasons. So, the Dems are scared they are missing something, and the Reps are going "Haha, who cares about modelling".

Will, the forecasts be perfect? Nope. But is the margin rather large, but not unsurmountable? Yes it is. Are the mistakes from last time repaired? Yes, they take care of uneducated whites. Is there evidence Trump has found a new source of voter support? I haven't seen it.

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#189

Post by rsynnott » Sun, Oct 25, 2020, 11:07 PM UTC

cwhiz wrote: post #28

Not sure if it’s wrong to put this here, but here is a link to their election forecast. https://projects.economist.com/us-2020-forecast/president You can compare this to the 538 model and see where these two teams and forecasts disagree.

Well, I don’t see how they expect anyone to take this seriously. No cartoon fox at all!

Re: Reverse-engineering the problematic tail behavior of Fivethirtyeight forecast

#190

Post by jeremyjh » Sun, Oct 25, 2020, 11:16 PM UTC

School-Cotton wrote: post #41

an_opabinia wrote: post #31

Andrew Gelman designed the 538 model in 2007. Nate Silver authored an adjustment to polls used in that model. Polls have more impact if they are more representative of statewide turnout among demographic things he chose like “black” and “low income.” This is why his predictions were so accurate for Obama’s 2008 and 2012 elections, and likely why they were so inaccurate in 2016. Gelman’s own grad student is the only p…

Nate Silver said Trump had a 1 in 3 chance, which basically means one shouldn’t be surprised no matter the result. I’m not sure where this “all the polls were so far off in 2016!!” narrative comes from, but it’s wrong.

That isn't the only thing that happened. Probably the 1 in 3 odds were too low given the data available, because the polling demographics were not adjusted for education. If you randomly sample 1000 people to represent several millions, you also collect demographic information to ensure that you properly weight the responses based on how skewed that demographic is in your sample compared to the total voting population. In 2016 they weren't correcting for education, which turned out to be a huge hidden variable. This is explained quite well by 538 themselves: https://fivethirtyeight.com/videos/polling-101-what-happened...