Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

towardsdatascience.com

Post reply on HN
Print view View original

131–140 of 212 posts

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#131

Post by CompelTechnic »

duckerude wrote: post #113
robbiep wrote: post #66

Earlier quoted context omitted.

That might be really useful for you, but for those of us who don’t design nuclear reactors it is utterly meaningless - what are the consequences of each of the results and how much should I care about them? I would be blown away if more than 1 in 10,000 could understand what the consequences are of your statement. Care to share?

If I'm understanding it correctly, aleatory uncertainty is uncertainty you understand, and epistemic uncertainty is uncertainty due to lack of knowledge. Uncertainty about the outcome of an unbiased die roll is aleatory. You don't know what's going to happen, but that's because of inherent uncertainty, not because you don't know enough about the die. So while you can't be sure what the outcome will be, you can know t…

To continue the dice analogy-

I can imagine a world where gambling sharks arrive at the roulette wheel in casinos, wearing hidden google-glass-ish devices, and place bets while the ball is in motion. Unbeknownst to everyone else, their device performed real time computation based on the position and velocity of the ball and wheel, and gave an output of a tight probability distribution (10% chance 25 red, 35% chance 29 black, 40% chance 12 red, 10% chance 8 black, 5% other). The sharks place bets based on these distributions, and make a bunch of money over the course of a few games.

This raises the questions:

1. To the public (not the sharks), all of the possible outcomes of the game have equal probability. Would it be correct to say that all of their uncertainty regarding the outcome of the roulette wheel is aleatory?

2. To the card sharks, they DO have a model with knowledge of game outcomes. So should one say that the distribution output by the device is aleatory uncertainty, and the remaining uncertainty (the fact that there is a distribution instead of a single predicted outcome) in the system is epistimic?

3. #1 and #2 differ in presuming that such a device is present. In absense of such a device, or in general, in absense of a tested model, is it correct to say that all uncertainty is epistimic?

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#132

Post by objektif »

darawk wrote: post #64
comex wrote: post #41

Earlier quoted context omitted.

I don’t have the background knowledge to properly understand the math in that paper. But I do know that you can’t simply set bounds a priori on how much a probability can change after additional information has been gathered; no matter how pathological the swings, you can design a system, and a series of observations of that system, that would make all the forecasts (i.e. conditional probabilities) correct. Thus the…

I believe he's making a slightly more subtle point. It's not just that the forecasts are swinging too much, it's that they're swinging too much too early. Consider an option on a stock (which is the analogy Taleb is making here). If you buy a 1 year call option on AAPL and tomorrow they announce that they beat earnings by 10%, that's not a huge deal for you. If on the other hand, you owned a 1-week expiration call, i…

That would be only true if the thing you are forecasting becomes more sensitive to changes in the environment as time passes. It is true in your option example but not entirely true for elections.

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#133

Post by lftl »

OscarCunningham wrote: post #96
safgasCVS wrote: post #94

Earlier quoted context omitted.

Unlikely. Taleb seems to like how his argument has been summarised. The crux of the issue taleb has with silver is showing probabilities without making a decision/declaration is cowardly. When asked about making a prediction he says one thing then follows it up with a “ but don’t be surprised if it could be completely random.” And that’s the point - the fact he covers his ass is the problem. Election forecasting is h…

I disagree. Probabilities can be useful even if they're not 0% or 100%. For example weather forecasts often give the probability of rain. My plans will be different if the probability is 20% vs 80%. My plans don't tend to vary based on who wins the election, but I imagine that that information is very useful to some people. And if your data is telling you to be 80% sure, I don't think you should fudge it and claim 10…

With the weather man, I could keep track of when it rains and doesn't. If it rains significantly more or less than 80% of the time they say there's an 80% chance of rain, I can say that their model is bad.

I can't think of a similar way to evaluate Silver's election forecasting model. They very clearly aren't independent probabilities, and his model changes significantly from cycle to cycle. Was his model good in 2012 when every state went to where he predicted the likely probability was? Was it bad when his model said Hillary had a 71.4% chance of winning?

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#134

Post by mannykannot »

logifail wrote: post #76
ummonk wrote: post #72

Earlier quoted context omitted.

I normally like Taleb's insights but it was obvious when following this that he (and Dinesh D'Souza) were ridiculously misconstruing Nate Silver's comment. Two outcomes being "extremely possible" in no way means they have a 50/50 chance of happening. It merely means that neither outcome has a low enough chance that you should assume it won't happen.

>> Two outcomes being "extremely possible" What on earth is a professional like Silver doing using the word "extremely" in that context? If I'd have written that kind of statement in school, the teacher would have crossed it out in red pen and told me to rephrase. If I toss a coin, are the outcomes "heads" and "tails" both supposed to be "extremely possible", with "landing on its edge" being merely "unlikely"? (In ca…

> What on earth is a professional like Silver doing using the word "extremely" in that context?

Extemporizing while being interviewed on TV, presumably to stress the uncertainty in the prediction, is what he is doing, not writing an HN post, tweet or paper.

I suppose that whenever you are interviewed on national TV, you only say exactly what you mean, nothing more and nothing less?

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#135

Post by tnzn »

OscarCunningham wrote: post #96
safgasCVS wrote: post #94

Earlier quoted context omitted.

Unlikely. Taleb seems to like how his argument has been summarised. The crux of the issue taleb has with silver is showing probabilities without making a decision/declaration is cowardly. When asked about making a prediction he says one thing then follows it up with a “ but don’t be surprised if it could be completely random.” And that’s the point - the fact he covers his ass is the problem. Election forecasting is h…

I disagree. Probabilities can be useful even if they're not 0% or 100%. For example weather forecasts often give the probability of rain. My plans will be different if the probability is 20% vs 80%. My plans don't tend to vary based on who wins the election, but I imagine that that information is very useful to some people. And if your data is telling you to be 80% sure, I don't think you should fudge it and claim 10…

But do weather forecast always end with "but this could also be completely random" to avoid being accountable ?

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#136

Post by zellyn »

ubernostrum wrote: post #121
zimablue wrote: post #87

To try and explain Taleb's argument as simply as possible: If someone tells you today that an event is 100% certain, and tomorrow tells you that it is impossible, then you intuitively will not trust their forecasting. If it's certain, then it's certain not to be impossible tomorrow Now if someone tells you something is a 90% chance today, and a 10% chance tomorrow, again you don't trust their predictions. There is a…

I've always treated 538's forecasts as meaning nothing more or less than "We have a model, we fed the current data into it and ran N simulations of the event, and in X% of those simulations, this was the result". Which, as I understand it, is literally how they're generated. Any discussion of the accuracy of the model (which should be the only thing that admits debate) has to be retrospective. Just saying "well the f…

I came here to post this exact comment — except less eloquently. Hi, ubernostrum! :-)

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#137

Post by mannykannot »

safgasCVS wrote: post #94
mturmon wrote: post #25

This is an intellectually confused post. It is so unclear and muddled that only a charitable reader would suppose the author understands the mathematical and modeling issues here. I’m surprised the author published it in this form - you don’t want to jump into a controversy like this with such an unclear discussion. As just one example, the whole digression on a “decision boundary” is conceptually mistaken. Once you…

Unlikely. Taleb seems to like how his argument has been summarised. The crux of the issue taleb has with silver is showing probabilities without making a decision/declaration is cowardly. When asked about making a prediction he says one thing then follows it up with a “ but don’t be surprised if it could be completely random.” And that’s the point - the fact he covers his ass is the problem. Election forecasting is h…

When reporting on the run-up to an election, taking great care to avoid having one's figures misconstrued as suggesting that the outcome is settled, when it is not, is the responsible thing to do. For Silver to not say a damn thing would leave all the interminable speculation in the mouths of those with even less knowledge of what might happen, and those with an agenda.

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#138

Post by lavrov »

I thought that the post was fairly unclear, but for me, it seems like the main argument against 538 is that it makes unfalsifiable claims about the probability of individual elections - picking a winner is a falsifiable claim, but assigning a probability always allows Silver to claim something like “even events with a 10% probability occur frequently” even if his model assigns a high likelihood of victory to the loser of an election.

My favorite Nate Bronze takedown remains Carl Diggler[1].

[1] https://www.washingtonpost.com/amphtml/posteverything/wp/201...

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#139

Post by PeterisP »

safgasCVS wrote: post #94
mturmon wrote: post #25

This is an intellectually confused post. It is so unclear and muddled that only a charitable reader would suppose the author understands the mathematical and modeling issues here. I’m surprised the author published it in this form - you don’t want to jump into a controversy like this with such an unclear discussion. As just one example, the whole digression on a “decision boundary” is conceptually mistaken. Once you…

Unlikely. Taleb seems to like how his argument has been summarised. The crux of the issue taleb has with silver is showing probabilities without making a decision/declaration is cowardly. When asked about making a prediction he says one thing then follows it up with a “ but don’t be surprised if it could be completely random.” And that’s the point - the fact he covers his ass is the problem. Election forecasting is h…

If asked about the outcome of some uncertain future event (say, an election or some sports game), saying "team A has a 20% chance of winning" is making a prediction.

Making a decision/declaration in such a case where the outcome isn't (yet) certain does not mean "being held accountable", it means that you're either stupid, or a liar, or a stupid liar. If you want to call the results of a game before it's certain, then you shouldn't say a damn thing, because anything you say is a lie if you're falsely implying that the result is certain.

If reality is uncertain, then any certain statements/predictions are by definition wrong. Some predictions have more certainty than others, you can stake things on such predictions (sports betting is a great example - if you think that there's a 20% chance of winning but others think that it's 10% or 30%, then there's an opportunity), but it's ridiculous to require certainty where certainty shouldn't be expected.

Post reply on HN