Why we should care about the Nate Silver vs. Nassim Taleb Twitter war • Hacker News Board

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#111

Post by OscarCunningham » Tue, Dec 18, 2018, 12:15 PM UTC

zimablue wrote: post #109

OscarCunningham wrote: post #90

Earlier quoted context omitted.

> Now Nate Silver accepts this, but says that he's not actually putting a probability on the event in the future, but some non-existent "probability on that event in the future if the future was now". Could you link to where Silver discusses this? I'm interested in seeing his description of exactly what his numbers mean.

https://fivethirtyeight.com/features/a-users-guide-to-fiveth... What I'm referring to is the "now-cast", but his other two definitions both seem to shy away from saying "this is flat-out the probability we think of the election". The point is, you can redefine or choose a definition of probability if you want, but if it's less useful than the normal definition (and confusing to people!) then people are free to critic…

Oh okay. I think it's fair to just take his polls-plus model as his prediction and ignore the now-cast. But I wouldn't say that showing the now-cast is somehow being sneaky. He's just providing extra information.

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#112

Post by safgasCVS » Tue, Dec 18, 2018, 12:16 PM UTC

OscarCunningham wrote: post #96

safgasCVS wrote: post #94

Earlier quoted context omitted.

Unlikely. Taleb seems to like how his argument has been summarised. The crux of the issue taleb has with silver is showing probabilities without making a decision/declaration is cowardly. When asked about making a prediction he says one thing then follows it up with a “ but don’t be surprised if it could be completely random.” And that’s the point - the fact he covers his ass is the problem. Election forecasting is h…

I disagree. Probabilities can be useful even if they're not 0% or 100%. For example weather forecasts often give the probability of rain. My plans will be different if the probability is 20% vs 80%. My plans don't tend to vary based on who wins the election, but I imagine that that information is very useful to some people. And if your data is telling you to be 80% sure, I don't think you should fudge it and claim 10…

But look at what you did - you looked at the prob and then made a decision. And that’s kinda Talebs whole spiel - quoting probabilities is moot without an associated action else its just masturbating for attention. Nate shows the probabilities and puts his hands up and says well it could be anything etc.

It’s reminds one of those pundits who are bullish on a stock but won’t put their money where their mouths are. If you the viewer buy it because of their recommendations and they are right then they are happy to get the praise but if it goes wrong and you the viewer pay the price they get to say “well the market is highly uncertain etc”. If the guy won’t eat his own dog food then why should others etc

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#113

Post by duckerude » Tue, Dec 18, 2018, 12:21 PM UTC

robbiep wrote: post #66

acidburnNSA wrote: post #39

Earlier quoted context omitted.

Peripherally, I like using nuclear reactor design analogies for Aleatory vs. Epistemic. Aleatory is the inherent randomness of a system. You can characterize it but it's often hard to reduce. For example, how much boron impurities are mixed into your steel at the time of manufacture. Epistemic is things that are knowable but you don't know with much certainty (because it's hard to measure), like the probability that…

That might be really useful for you, but for those of us who don’t design nuclear reactors it is utterly meaningless - what are the consequences of each of the results and how much should I care about them? I would be blown away if more than 1 in 10,000 could understand what the consequences are of your statement. Care to share?

If I'm understanding it correctly, aleatory uncertainty is uncertainty you understand, and epistemic uncertainty is uncertainty due to lack of knowledge.

Uncertainty about the outcome of an unbiased die roll is aleatory. You don't know what's going to happen, but that's because of inherent uncertainty, not because you don't know enough about the die. So while you can't be sure what the outcome will be, you can know that each outcome has a 1/6 probability, and you shouldn't expect to adjust that probability as you gain more knowledge.

But uncertainty about the outcome of an election is epistemic, because it depends on a lot of present unknowns. If you run more polls you could reach a different, more accurate probability. Part or all of the uncertainty is because of things you don't know.

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#114

Post by mijamo » Tue, Dec 18, 2018, 12:23 PM UTC

dagw wrote: post #78

blurbleblurble wrote: post #74

I'm still waiting for somebody to talk about how tightly coupled Nate Silver's forecasts are to the political processes he attempts to forecast. What if Nate Silver were to run around on the football field before each play whispering in the players' ears about the how likely they are to win, according to his latest forecast. Imagine that they believed in him, or at least felt some degree of superstition when he came…

I just think there's something innately flawed about public forecasting of an election. It used to be the case in France that you weren't allowed to publish polls two weeks before an election for just this reason. They eventually scrapped this with the argument that the all 'elites' had access to internal and private polls anyway so all you where doing was denying the 'people' the same access to information that the…

This is slightly wrong. It was initially one week, not two. It was changed because it was unconstitutional, as it limited freedom of expression too much, not because it was useless or anything to do with elite vs people. And it still exists, limited to the day of the election and the day before. Source: https://www.conseil-constitutionnel.fr/nouveaux-cahiers-du-c...

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#115

Post by dagw » Tue, Dec 18, 2018, 12:23 PM UTC

lixtra wrote: post #110

dagw wrote: post #100

Earlier quoted context omitted.

Best solution proposed here So what should the bet be? As far as I can tell they're not disagreeing on anything that can easily be reduced to a bet

Nate calls the chances like 80:20 Clinton:Trump. Taleb takes the side: 100,000 USD that Trump wins. If Trump wins, Nate pays 500,000 to Taleb. If Clinton wins, than Taleb pays 100,000 USD to Nate. Do this for each election.

But that doesn't seem to be the nature of their disagreement. Taleb's main complaint seems to be about how the swings in the prediction probabilities far out from the event indicates a fundamental flaw in how the underlying model uses probability.

That being said it would be a fun little exercise to see who much money would be made if they bet had $1 on every race 538 predicted in 2016 and 2018 using the model odds.

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#116

Post by bryanrasmussen » Tue, Dec 18, 2018, 12:23 PM UTC

coldtea wrote: post #67

hn_throwaway_99 wrote: post #37

Earlier quoted context omitted.

Agreed. The more I read the more I got embarrassed for the author, mainly because his points were so ridiculously flimsy. I thought this was one of the worst: > Because FiveThirtyEight only predicts probabilities, they do not ever take an absolute stand on an outcome: No ‘skin in the game’ as Taleb would say. This is not, however, something their readers follow suit on. In the public eye, they (FiveThirtyEight) are j…

> Saying "this is not something their readers follow suit on" - only if you're a reader who doesn't understand probability Isn't that the very readership they capitalize on?

As said he tries to help his readership understand how probability works, but in the end one may still ethically do business in one's chosen field without having to solve the problems of understanding the limits of that field for those uneducated in it.

on edit: grammar was atrocious, still bad now.

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#117

Post by circlefavshape » Tue, Dec 18, 2018, 12:24 PM UTC

safgasCVS wrote: post #94

mturmon wrote: post #25

This is an intellectually confused post. It is so unclear and muddled that only a charitable reader would suppose the author understands the mathematical and modeling issues here. I’m surprised the author published it in this form - you don’t want to jump into a controversy like this with such an unclear discussion. As just one example, the whole digression on a “decision boundary” is conceptually mistaken. Once you…

Unlikely. Taleb seems to like how his argument has been summarised. The crux of the issue taleb has with silver is showing probabilities without making a decision/declaration is cowardly. When asked about making a prediction he says one thing then follows it up with a “ but don’t be surprised if it could be completely random.” And that’s the point - the fact he covers his ass is the problem. Election forecasting is h…

Cowardly? He's not "covering his ass", he's doing statistics

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#118

Post by dmurray » Tue, Dec 18, 2018, 12:29 PM UTC

OscarCunningham wrote: post #101

darawk wrote: post #64

Earlier quoted context omitted.

I believe he's making a slightly more subtle point. It's not just that the forecasts are swinging too much, it's that they're swinging too much too early. Consider an option on a stock (which is the analogy Taleb is making here). If you buy a 1 year call option on AAPL and tomorrow they announce that they beat earnings by 10%, that's not a huge deal for you. If on the other hand, you owned a 1-week expiration call, i…

I think that Taleb's problem is that he thinks all the uncertainty in Silver's model comes from the fact that people might change their minds about who to vote for. This is why Taleb can't make a model that fits Silver's forecasts. Silver could only be confident early on if he knew that people weren't going to change their minds much. But if people don't change their minds much then Silver's forecast shouldn't fluctu…

No, it's mostly caused by people changing their minds on who to vote for (or whether to vote). That's why the estimates move around a lot when there is news affecting the election, like the given example of Comey re-opening the Clinton investigation.

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#119

Post by mhb » Tue, Dec 18, 2018, 12:35 PM UTC

dagw wrote: post #115

lixtra wrote: post #110

Earlier quoted context omitted.

Nate calls the chances like 80:20 Clinton:Trump. Taleb takes the side: 100,000 USD that Trump wins. If Trump wins, Nate pays 500,000 to Taleb. If Clinton wins, than Taleb pays 100,000 USD to Nate. Do this for each election.

But that doesn't seem to be the nature of their disagreement. Taleb's main complaint seems to be about how the swings in the prediction probabilities far out from the event indicates a fundamental flaw in how the underlying model uses probability. That being said it would be a fun little exercise to see who much money would be made if they bet had $1 on every race 538 predicted in 2016 and 2018 using the model odds.

It doesn't need to be a simple bet. It could incorporate the volatility which Taleb might believe shouldn't be present in an unflawed model.

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#120

Post by FactolSarin » Tue, Dec 18, 2018, 12:40 PM UTC

paganel wrote: post #62

VintageCool wrote: post #58

Earlier quoted context omitted.

The 2016 election was swingy and it was uncertain. In 2008 and 2012, the political media was myopically focused on the twists and turns of the horse race and called it 50-50 dead even. Nate Silver's 538 modeled the election as basically stable with only small polling shifts, and the only change in the last month was a steady decline in the remaining time that McCain (or Romney) had to significantly shift the polls. I…

>The 2016 election was swingy and it was uncertain. I don't remember it that way, in fact I remember seeing a lot of very shocked faces in the Democratic camp when the results were beginning to take shape. I don't remember having seen similar confused reactions after any previous US presidential elections, not even after the Bush vs Gore one which was a lot closer in terms of electoral votes. Back to the article, I t…

However, FiveThirtyEight gave Trump a 30% chance of winning, and in the days before the election was stressing the fact that a Trump win was very possible.

They didn't predict his win, but they did better than anyone else in the mainstream.