Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

towardsdatascience.com

Post reply on HN
Print view View original

181–190 of 212 posts

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#181

Post by zimablue »

thousandautumns wrote: post #175
zimablue wrote: post #82

Earlier quoted context omitted.

I think he would say that the "nowcast" is so useless and deceptive to be malpractice, since everyone will interpret it as a forecast, and even if they didn't it's not in any way useful. Which I agree with (unsurprising since I've semi-invented it as an opinion for another person).

It's only useless and deceptive if you willfully do absolutely nothing to understand what it is, or how it differs from the other models 538 has, which apparently is the case for Taleb.

Or in other words, you're 99% of people who will ever hear the number

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#182

Post by coryfklein »

Kurtz79 wrote: post #177
hn_throwaway_99 wrote: post #149

Earlier quoted context omitted.

Fine, but Silver goes to extreme ends in my opinion to explain what a probability means. During the 2016 presidential election he was emphasizing "Hey, it's less likely than the alternative, but there is a real possibility Trump will win. There is a very plausible path for him to wrap up the electoral votes." And afterwards, I think the results showed how accurate Silver was - he gave a much higher probability to Tru…

I was following the website frequently at the time and I fully agree. Bur I was just commenting that that the author is spot-on at least in that many readers do not take these explanations into account and they see just 28.6% of winning chance and they automatically convert it to 0% in their mind. And the explanation is that they don't understand math and probability (or don't want to), whether the author likes it or…

I still don't get why the author thinks it's OK to assign blame to 538 for readers interpreting things incorrectly when Nate clearly spends a significant amount of time trying to educate the readers on how to interpret.

It makes no sense to me to hold 538 accountable for readers interpretations of the data. If I wander into some realm of science I'm unfamiliar with (say, quantum mechanics) and start making false inferences from it, does that mean the scientists were misleading me?

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#183

Post by paganel »

FactolSarin wrote: post #120
paganel wrote: post #62

Earlier quoted context omitted.

>The 2016 election was swingy and it was uncertain. I don't remember it that way, in fact I remember seeing a lot of very shocked faces in the Democratic camp when the results were beginning to take shape. I don't remember having seen similar confused reactions after any previous US presidential elections, not even after the Bush vs Gore one which was a lot closer in terms of electoral votes. Back to the article, I t…

However, FiveThirtyEight gave Trump a 30% chance of winning, and in the days before the election was stressing the fact that a Trump win was very possible. They didn't predict his win, but they did better than anyone else in the mainstream.

> They didn't predict his win, but they did better than anyone else in the mainstream.

Kind of proving my point, as it shows that Nate Silver was the best at crystal-balling the election result. Afaik 30% is still bellow the 50% (or 0.5/1) threshold generally needed to make a decision, as the article also mentions.

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#184

Post by coryfklein »

safgasCVS wrote: post #179
PeterisP wrote: post #139

Earlier quoted context omitted.

If asked about the outcome of some uncertain future event (say, an election or some sports game), saying "team A has a 20% chance of winning" is making a prediction. Making a decision/declaration in such a case where the outcome isn't (yet) certain does not mean "being held accountable", it means that you're either stupid, or a liar, or a stupid liar. If you want to call the results of a game before it's certain, the…

This is not what the disagreement is about. It’s got nothing to do with uncertainty in the predictions and the debate is not about saying for certain what is going to happen. We know that distributions have means and variances so one can’t say what WILL happen. The point taleb is making is that Nate presents likelihoods and then throws his hands up and says whoa there I’m not saying what is going to happen just look…

I think this is a mischaracterization of Nate's own words. He certainly stakes his own reputation on the accuracy of his forecast. They spend a lot of time looking back and seeing how accurate the model was for the election as a whole.

There are two kinds of things Nate is saying here, and they are related but distinct:

1. Educating readers/listeners about how 20% chances happen all the time 2. Internal analysis and external reporting of how often his 20% predictions were right. If 538 predicts a group of 100 congressmen to all have a 20% chance of being elected and then 21 do get elected, the model did well and Nate's work is worth money to ABC and his reputation is improved (or maintained). If instead 33 of those congressmen get elected, the opposite result happens.

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#185

Post by huffmsa »

mercutio2 wrote: post #170
huffmsa wrote: post #141

Earlier quoted context omitted.

Yes but that's not how the proles understand the blog. They take it as "x has a y % chance of happening". Which is Taleb's beef with it. The commoner thinks its Silver putting his money on something, but when it doesn't happen he says "well I didn't say _that_". And it in the day and age of more robust machine learning models, that the 538 models are so volatile is a pretty weak excuse.

Humans are volatile. Politics even more so. Our options aren’t “high confidence converging predictions, or 538”. The options in the current era for understanding of the electorate’s mood are “overly confident individual polls, poorly analyzed with completely inadequate models”, vs “538-style epistemically humble models accompanied by discussions of their confidence, which can be scored in aggregate after each electio…

But it's not about you. Just being on this forum puts you in the critical thinking 1%.

538 fails in that most people think that the daily stats are Silver's betting positions. He "predicted the 2008 election."

We understand the difference, but the crying campaign staffers last November did not.

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#186

Post by coryfklein »

zimablue wrote: post #87

To try and explain Taleb's argument as simply as possible: If someone tells you today that an event is 100% certain, and tomorrow tells you that it is impossible, then you intuitively will not trust their forecasting. If it's certain, then it's certain not to be impossible tomorrow Now if someone tells you something is a 90% chance today, and a 10% chance tomorrow, again you don't trust their predictions. There is a…

Could you use the same reasoning to discount weather predictions?

September 10, 2018 - Hurricane Florence predicted to be category 4 hurricane on landfall, 80% Septemeber 12, 2018 - Hurricane Florence predicted to be category 4 hurricane on landfall, 20% September 14, 2018 - Hurricane Florence makes landfall as Category 1 hurricane

(the above numbers are demonstrative, loosely based on my memory of how events actually happened with Florence)

There is something to be said for predictions about the future based on today's environment, while still allowing for the reality that the environment could change. Predicting a single baseball game right before it happens is just a different kind of prediction than simulating a model that has noisy cross-interacting inputs.

Readers/listeners of 538 need to understand (and Nate spends a lot of time educating about this) exactly what the model is calculating and what it isn't. Nate calls out all the time that the model can only be as good as the polling that provides the inputs. And polls can swing for all sorts of reasons: there's not many of them for a district, only highly biased ones are available, people's actual voting intentions change from week to week.

Am I missing the point of what you're trying to say?

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#187

Post by VintageCool »

coldtea wrote: post #71
VintageCool wrote: post #58

Earlier quoted context omitted.

The 2016 election was swingy and it was uncertain. In 2008 and 2012, the political media was myopically focused on the twists and turns of the horse race and called it 50-50 dead even. Nate Silver's 538 modeled the election as basically stable with only small polling shifts, and the only change in the last month was a steady decline in the remaining time that McCain (or Romney) had to significantly shift the polls. I…

> The 2016 election was swingy and it was uncertain. In retrospect. In real life, only a few pundits called it from the start (even explaining the mechanics, including Taleb and that Dilbert guy). Most of the press and pundits was all certain about a specific outcome until the very last minute.

The Real Clear Politics polling aggregator showed nearly a 10 point jump for Hillary Clinton after "grab em by the pussy" came out. That lead then eroded back to the mean over the course of the month.

That's what I mean by swingy. Nothing in 2008 or 2012 shifted the polls like that. The polls in those elections stayed within a 2 point band the whole way. That level of volatility was clear in the polls well in advance of the election.

(And, 15% of the electorate was effectively undecided with 2 weeks to go, which is much higher than it was in 2008 or 2012).

Such swings are also what Taleb and the author of this article are criticizing Nate Silver for.

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#188

Post by coryfklein »

Re: the desire to choose a 50% (or other) decision threshold.

It doesn't have to be a binary choice. Knowing the distinction between 65% and 80% improves your betmaking capabilities by allowing you to hedge, in some cases guaranteeing a profit [1] regardless of the odds and how they change.

[1] https://help.smarkets.com/hc/en-gb/articles/115001431011-How...

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#189

Post by leereeves »

hn_throwaway_99 wrote: post #149
Kurtz79 wrote: post #63

Earlier quoted context omitted.

"Saying "this is not something their readers follow suit on" - only if you're a reader who doesn't understand probability." Unfortunately that is the more common case, and I agree on the author on that that most people take a binary stance on polls. It should not be the case for regular readers of a website whose main topic is statistical analysis, although I imagine that just before a major election they would see a…

Fine, but Silver goes to extreme ends in my opinion to explain what a probability means. During the 2016 presidential election he was emphasizing "Hey, it's less likely than the alternative, but there is a real possibility Trump will win. There is a very plausible path for him to wrap up the electoral votes." And afterwards, I think the results showed how accurate Silver was - he gave a much higher probability to Tru…

Nate Silver didn't always hedge his bets; early in the election he was making emphatic (and wrong) predictions like these:

> The lesson, rather, is that Trump’s campaign will fail by one means or another.

> Donald Trump Is Winning The Polls — And Losing The Nomination

> Our emphatic prediction is simply that Trump will not win the nomination. It’s not even clear that he’s trying to do so.

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#190

Post by leereeves »

scott_s wrote: post #161
hn_throwaway_99 wrote: post #149

Earlier quoted context omitted.

Fine, but Silver goes to extreme ends in my opinion to explain what a probability means. During the 2016 presidential election he was emphasizing "Hey, it's less likely than the alternative, but there is a real possibility Trump will win. There is a very plausible path for him to wrap up the electoral votes." And afterwards, I think the results showed how accurate Silver was - he gave a much higher probability to Tru…

I agree. I think Silver laid it out clearly in a piece that people called him a troll for writing: Election Update: Yes, Donald Trump Has A Path To Victory, https://fivethirtyeight.com/features/election-update-yes-don... . Another political writer at 538, Harry Enten, had a similar story a few days later, Trump Is Just A Normal Polling Error Behind Clinton, https://fivethirtyeight.com/features/trump-is-just-a-normal-…

That's not the whole picture. Before he learned to be more careful, Silver was calling Trump a troll, doomed to fail, who wasn't even trying to win.

https://fivethirtyeight.com/features/donald-trump-is-winning...

https://fivethirtyeight.com/features/donald-trump-is-the-wor...

Post reply on HN