71–80 of 213 posts
Earlier quoted context omitted.
That seems like a pretty big difference to gloss over. Why are they publishing probabilities for if the event were to happen today? The event isn't happening today!
I know, right? Every time my meteorologist gives me a probability of rain on a day that it doesn't rain I ask the same question.
Isn’t it just one big misunderstanding between them? Taleb thinks 538’s probabilities represent a binary option price on the event, in which case, yes, the probabilities should stay very close to 50% because the vol is so high. Whereas Silver’s models are actually saying “Based on current polls, if the election were held tomorrow, then the probability candidate X wins is Y%” and are thus allowed to swing more wildly.…
But it's not a misunderstanding. Taleb understands Silver's approach, but thinks it's BS, entertainment not forecasting.
He's right, but I don't know why he cares. The problem isn't that Silver doesn't know math, it's that his goal is to entertain, not predict.
Taleb in person at a talk is very different than Taleb with a bottle of wine on Twitter. To both of their credit, they’ve expanded my knowledge of probability and statistics.
Earlier quoted context omitted.
Do you have an example of a time where Nate Silver has shown a misunderstanding of probability? Everything I've read by him has seemed informed.
The best criticism I can come up with is that 538 probably doesn't do a sufficient job of explaining to its audience that their published probabilities are predictions for what would happen if the event occurred today . This seems to be one of Taleb's (and others') criticisms of their predictions: that they don't consider the probability of major unexpected events occurring between now and the event (like an athlete…
I don't think that's true. While 538 does such predictions (the “nowcast”), they also predict actual results (the “polls-plus forecast” and the “polls-only forecast”), the default view is a forecast (I think the polls-plus usually, but I vaguely recall a chance to polls-only as the default during the 2016 cycle with concerns that there were reasons to suspect that some of the other signals on the polls-plus model had become less useful.) The “nowcast” is not the the default/headline number, so it would be very odd for a someone to mistakenly fixate on it thinking it was the forward-looking prediction.
Earlier quoted context omitted.
I have a hard time believing Taleb can keep any friends. He thinks everyone is an idiot, and doesn't have any reservation about telling them.
He's good friends with Stephen Wolfram. Take that any way you like.
Earlier quoted context omitted.
That seems like a pretty big difference to gloss over. Why are they publishing probabilities for if the event were to happen today? The event isn't happening today!
Because this is what every election poll does. Nobody asks people if they would vote in such person a year from now. They always ask what is your voting preference today. If Nate Silver did something different, then he would be completely wrong.
Exit polls are of course asking for voting preference today, because no person in their right mind would ever answer differently for "who would you vote for today?" versus "who will you vote for on November 4?" so it doesn't matter.
The problem is, we don't care about what would happen if the election were to be run today. We care about what will happen when the election actually occurs. What we want is a prediction that takes the polls into account but also factors in all the other information we have that could be used to try and predict the future election (e.g. trying to identify whether changes in a candidates polling numbers are likely to be temporary or reflect a permanent shift, identifying historical trends that can be used to predict future behavior, etc). I realize that predicting the future is much harder than predicting the hypothetical of what would happen today, but predicting the future is the whole point of forecasting.
There is value in being able to tease out the "accurate" voter sentiments today from the various polls and accounting for uncertainty and error, but that value is primarily in how we can then turn around and use that information to aid in future forecasting, and how candidates can use that data to gauge the effects of what they're doing. But without an attempt at forecasting the future, this is largely useless to the general population.
From a business perspective, the volatility of Silver's models seem like a feature - they enhance the drama / sensationalism of election coverage. One week, he's telling me candidate X will likely win, the next week it's candidate Y, and I'm on the edge of my seat. If Silver really believed these probabilities were correct, he should be willing make bets with these odds, otherwise his incentives are distorted.
Taleb's aim is something like predictive rigor. God knows why he cares about Silver. I think everyone gets that Silver is doing entertainment.
Earlier quoted context omitted.
Because this is what every election poll does. Nobody asks people if they would vote in such person a year from now. They always ask what is your voting preference today. If Nate Silver did something different, then he would be completely wrong.
If Nate Silver took the predictions for "if the event were to happen today" and pretended it was a prediction for the future event, he'd be wrong, but that's how most people interpret his predictions anyway so that makes him effectively wrong anyway. Exit polls are of course asking for voting preference today, because no person in their right mind would ever answer differently for "who would you vote for today?" vers…
The only thing these people need is to learn what he's really doing, and then he will be immediately correct again! To me this seems like the simple and obvious solution to this debate.