Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

towardsdatascience.com

Post reply on HN
Print view View original

81–90 of 212 posts

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#81

Post by dragonwriter »

013a wrote: post #45

The whole 2016 situation feels like many people are borderlining on delusion, Nate included. The Comey thing certainly helped Trump, but even the day of the election 538 was still forecasting a massive win for Hilary. How can the probabilities be that wrong? Because the polls were wrong. Or, at least, incomplete. It doesn't matter how good your algorithms are if the data isn't there. And 2016 proved that 538, and rea…

> but even the day of the election 538 was still forecasting a massive win for Hilary

It was predicting a large electoral win as the median scenario, but only a 70% chance of Clinton winning at all, because discrepancy between actual outcomes and polling were likely to be correlated across states, and many states were very close. This is where Silver differed from the other forecasts that were saying Clinton 90%+ based on the (historically false) idea that differences between votes and polls were independent across states.

> You can't possibly assemble all of the data necessary to be remotely confident in the probability of outcomes.

You often can be more than remotely confident, but it's true that the national result of the 2016 Presidential election wasn't one of those times. OTOH, Silver’s forecast reflected that, since ~70% is extremely low confidence.

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#82

Post by zimablue »

cm2012 wrote: post #8
pizza wrote: post #2

If 538 is trying to do aleatory probabilities, then it is kinda like a broken clock - right sometimes, but as a tool for answers about future events of any complexity (i.e. the real world, where actions have consequences, and errors can be grave..), I'd avoid putting one's confidence in something that claims it has a simple/'linear' relationship to the real future. It is also a cop out to claim "everybody who thinks…

Taleb is just mixing up the nowcast and the forecast, even after being corrected many times.

I think he would say that the "nowcast" is so useless and deceptive to be malpractice, since everyone will interpret it as a forecast, and even if they didn't it's not in any way useful. Which I agree with (unsurprising since I've semi-invented it as an opinion for another person).

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#83

Post by JadeNB »

scythe wrote: post #59
VintageCool wrote: post #55

Earlier quoted context omitted.

This "Twitter fight" began with Nate Silver talking to the media, trying to convey both his prediction and its uncertainty: https://thehill.com/homenews/sunday-talk-shows/414759-nate-s... > "So in the House we have Democrats with about a 4 in 5 chance of winning," Silver told ABC's "This Week." > "But no one should be surprised if they only win 19 seats and no one should be surprised if they win 51 seats," Silver add…

>In this article, the author notes that 538 only presents its model to readers in terms of probabilities, but the public tends to round their predictions up and down. The only way you could get "50-50" from that is if you read "the Democrats get 51 seats" as "the Democrats win the House", because without that inference, he didn't say anything about the chance that they win the House. He compared a very lopsided win t…

> He compared a very lopsided win to a slight loss, and those were equally likely.

I'm not arguing with you, since the one thing I know is that there's no surer way to making a mistake than jumping in confidently on probabilistic issues, especially issues of interpretation rather than of pure mathematics; but I think you must have misspoke, or I must have misunderstood. The quoted sentence says 'extremely probable' (an absolute condition), not 'equally likely' (a relative condition):

> "But no one should be surprised if they only win 19 seats and no one should be surprised if they win 51 seats," Silver added. "Those are both extremely possible, based on how accurate polls are in the real world."

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#84

Post by andrelaszlo »

mturmon wrote: post #57
nkurz wrote: post #46

Earlier quoted context omitted.

I think you may not be appreciating the subtlety of the repetition of the "probability of rolling a six". His claim is that aleatory probability starts with the assumption that you have a standard six-sided die with all sides weighted equally, but that epistemic uncertainty requires accounting for the uncertainty that you have a fair die, or even that it has six sides. So in both cases you are indeed trying to calcul…

Yes, agreed. I noticed that potential interpretation, mostly because I already knew the distinction. Now suppose you didn’t already know.

I didn't know and I had to read that part twice.

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#85

Post by _0ffh »

ummonk wrote: post #72
VintageCool wrote: post #55

Earlier quoted context omitted.

This "Twitter fight" began with Nate Silver talking to the media, trying to convey both his prediction and its uncertainty: https://thehill.com/homenews/sunday-talk-shows/414759-nate-s... > "So in the House we have Democrats with about a 4 in 5 chance of winning," Silver told ABC's "This Week." > "But no one should be surprised if they only win 19 seats and no one should be surprised if they win 51 seats," Silver add…

I normally like Taleb's insights but it was obvious when following this that he (and Dinesh D'Souza) were ridiculously misconstruing Nate Silver's comment. Two outcomes being "extremely possible" in no way means they have a 50/50 chance of happening. It merely means that neither outcome has a low enough chance that you should assume it won't happen.

Yeah, essentially "extremely possible" is just "far from impossible". Something can be very possible but still not very likely, surely?

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#86

Post by scythe »

JadeNB wrote: post #83
scythe wrote: post #59

Earlier quoted context omitted.

>In this article, the author notes that 538 only presents its model to readers in terms of probabilities, but the public tends to round their predictions up and down. The only way you could get "50-50" from that is if you read "the Democrats get 51 seats" as "the Democrats win the House", because without that inference, he didn't say anything about the chance that they win the House. He compared a very lopsided win t…

> He compared a very lopsided win to a slight loss, and those were equally likely. I'm not arguing with you, since the one thing I know is that there's no surer way to making a mistake than jumping in confidently on probabilistic issues, especially issues of interpretation rather than of pure mathematics; but I think you must have misspoke, or I must have misunderstood. The quoted sentence says 'extremely probable' (…

Not "extremely probable" -- it says "extremely possible", which is a malapropism or a new phrase. It's easy to mistake because it has no clear meaning. The comparison, though, does not appear to be D's win vs D's lose, but D's win by more than X vs D's lose, and the latter clearly (imo) suggests D is favored to win.

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#87

Post by zimablue »

To try and explain Taleb's argument as simply as possible: If someone tells you today that an event is 100% certain, and tomorrow tells you that it is impossible, then you intuitively will not trust their forecasting. If it's certain, then it's certain not to be impossible tomorrow

Now if someone tells you something is a 90% chance today, and a 10% chance tomorrow, again you don't trust their predictions.

There is a probability associated with "changes in probability". The probability of going from 100 to 0 should intuitively be 0%. The probability of going from 90 to 10 intuitively must be something like 20% (it's like 80 of the 90s "didn't happen", with a probability of 20).

A key point is that you can add this up every day, if something goes from 90->10->90 then it's even more unlikely than going 90->10.

So if you extend that intuition, put some real maths behind it, you can tell whether something is "likely to be a real probability" by the rate at which it changes. And if you have lots and lots of repeated predictions, you can be confident that something isn't a good probability (every timeseries has like a Now Nate Silver accepts this, but says that he's not actually putting a probability on the event in the future, but some non-existent "probability on that event in the future if the future was now". But that doesn't correspond to anything useful or measurable (it's untestable!!), and most people will assume it follows the normal meaning of probability, and it's honestly just silly.

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#88

Post by zimablue »

chimeracoder wrote: post #42
tareqak wrote: post #5

As a non-specialist, I like how the discussion boils down the picture at the end: https://cdn-images-1.medium.com/max/1600/1*D_tidaT-fHMY3DRLg... in that it is something I can understand.

That's a really misleading graph, for a couple of reasons: 1. It conflates the forecast with the now-cast. The former is a prediction of what will happen on Election Day (whether that day is six months away or one day away). The latter is a prediction of what would happen if the election were held that day. In theory, those two would converge to the same value on the day of the election, though in practice, that woul…

Your second point is kind of confused. You don't need the prediction to coincide with the event, to judge the quality of the prediction. If I predict the elections for 2 years time then you can still judge whether I got it right or wrong. If I do that for multiple elections for multiple cycles, you can decide whether I'm a good forecaster.

If I then make another prediction that's always 1 year out, you can decide how good that prediction is and start to make a curve of how good my predictions are by how far out they are.

Taleb is saying that the blue line is so incorrect it's meaningless and deceptive and Nate should product a red line which is a real probability that he can actually be judged on.

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#89

Post by YeGoblynQueenne »

mturmon wrote: post #25

This is an intellectually confused post. It is so unclear and muddled that only a charitable reader would suppose the author understands the mathematical and modeling issues here. I’m surprised the author published it in this form - you don’t want to jump into a controversy like this with such an unclear discussion. As just one example, the whole digression on a “decision boundary” is conceptually mistaken. Once you…

>> Source: do UQ in day job.

UQ: Uncertainty Quantification.

(Not, say, Universal Quantifiers, etc).

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#90

Post by OscarCunningham »

zimablue wrote: post #87

To try and explain Taleb's argument as simply as possible: If someone tells you today that an event is 100% certain, and tomorrow tells you that it is impossible, then you intuitively will not trust their forecasting. If it's certain, then it's certain not to be impossible tomorrow Now if someone tells you something is a 90% chance today, and a 10% chance tomorrow, again you don't trust their predictions. There is a…

> Now Nate Silver accepts this, but says that he's not actually putting a probability on the event in the future, but some non-existent "probability on that event in the future if the future was now".

Could you link to where Silver discusses this? I'm interested in seeing his description of exactly what his numbers mean.

Post reply on HN