Nassim Talebs case against Nate Silver is bad math • Hacker News Board

Re: Nassim Talebs case against Nate Silver is bad math

#111

Post by dragonwriter » Tue, Apr 09, 2019, 9:22 PM UTC

Someone1234 wrote: post #89

tcbawo wrote: post #62

Earlier quoted context omitted.

Loads of people criticized Silver for not having "predicted" Trump's electoral victory. I imagine Silver's language was trying to hand-hold people to an understanding that even 20% probability events happen 1 out of 5 times. With so many people trying to twist and spin every headline, prediction, number, I appreciate Silver's attempt to make a reasonable guess with transparent methodologies.

It is worth noting that 538 doesn't keep a historical record of their predictions, making it difficult to really see how they predict outcomes coming up to a conclusion. You can only see the latest/final prediction. 538 also predicted lower chances of victory in the weeks and months leading up to that, then adjusted it on election day vanishing away their earlier guesses. This isn't wrong on the face of it, but peopl…

> It is worth noting that 538 doesn't keep a historical record of their predictions, making it difficult to really see how they predict outcomes coming up to a conclusion. You can only see the latest/final prediction.

It's worth noting that you are completely wrong: not only do they keep such a record, they also provide a handy chart of it on the latest prediction page so you can easily see exactly how the prediction changed over time.

Re: Nassim Talebs case against Nate Silver is bad math

#112

Post by ikeboy » Tue, Apr 09, 2019, 9:24 PM UTC

galby wrote: post #108

ikeboy wrote: post #67

Earlier quoted context omitted.

Well obviously Silver isn't actually offering to bet at those odds. But complaining that a model is incorrect because it enables arbitrage is a standard complaint, it's how you can prove all non-Bayesian models are irrational. That doesn't mean all Bayesian ones automatically pass the test though. Error bars and spreads are already outside the Bayesian paradigm, which demands a single number.

Can you expand on this? Bayesian data analysis looks at the full posterior, and reducing that to a single number is frowned upon by most.

If you're looking at a single proposition then the probability of it being true is a single number.

To say that the probability of it being true is a range is incoherent.

Re: Nassim Talebs case against Nate Silver is bad math

#113

Post by dragonwriter » Tue, Apr 09, 2019, 9:25 PM UTC

dandelany wrote: post #97

Someone1234 wrote: post #89

Earlier quoted context omitted.

It is worth noting that 538 doesn't keep a historical record of their predictions, making it difficult to really see how they predict outcomes coming up to a conclusion. You can only see the latest/final prediction. 538 also predicted lower chances of victory in the weeks and months leading up to that, then adjusted it on election day vanishing away their earlier guesses. This isn't wrong on the face of it, but peopl…

Maybe this was true of past elections, but their 2018 house forecast had a “how this forecast has changed” section that showed their historical predictions throughout election season. It’s still visible here: https://projects.fivethirtyeight.com/2018-midterm-election-f...

> Maybe this was true of past elections, but their 2018 house forecast had a “how this forecast has changed” section that showed their historical predictions throughout election season.

It has not changed. They did this the whole way back when it was an independent blog before the NY Times (later ESPN then ABC News) picked them up.

Re: Nassim Talebs case against Nate Silver is bad math

#114

Post by [deleted] » Tue, Apr 09, 2019, 9:26 PM UTC

throwawaymath wrote: post #46

LanguageGamer wrote: post #34

From a business perspective, the volatility of Silver's models seem like a feature - they enhance the drama / sensationalism of election coverage. One week, he's telling me candidate X will likely win, the next week it's candidate Y, and I'm on the edge of my seat. If Silver really believed these probabilities were correct, he should be willing make bets with these odds, otherwise his incentives are distorted.

> If Silver really believed these probabilities were correct, he should be willing make bets with these odds, otherwise his incentives are distorted. Yes, the "skin in the game" argument. That is Nassim Taleb's calling card. Personally, I consider it a little unreasonable to expect people to literally put their money where their mouth is every time they make a forecast.

[deleted]

Re: Nassim Talebs case against Nate Silver is bad math

#115

Post by plainOldText » Tue, Apr 09, 2019, 9:28 PM UTC

andrewla wrote: post #14

As I understand it, the feud in its current form started from a tweet from Dinesh D'Souza [1] in which he interpreted Nate Silver's statement that "Dems, GOP winning House are 'both extremely possible'" to mean that they had gone from an 80% chance of winning to a 50% chance. Taleb inexplicably chose this as the hill to die on [2] and wrote a lengthy and technical paper to refute, and replied (on the [2] thread) "Whe…

Here's another article analyzing their disagreement: https://towardsdatascience.com/why-you-should-care-about-the...

Re: Nassim Talebs case against Nate Silver is bad math

#116

Post by noelsusman » Tue, Apr 09, 2019, 9:29 PM UTC

baddox wrote: post #27

brian_cloutier wrote: post #16

Earlier quoted context omitted.

Do you have an example of a time where Nate Silver has shown a misunderstanding of probability? Everything I've read by him has seemed informed.

The best criticism I can come up with is that 538 probably doesn't do a sufficient job of explaining to its audience that their published probabilities are predictions for what would happen if the event occurred today . This seems to be one of Taleb's (and others') criticisms of their predictions: that they don't consider the probability of major unexpected events occurring between now and the event (like an athlete…

That's just not true. In 2016 they had several versions of the model. One was called the NowCast that very explicitly modeled the probability of winning the election today. They had a separate model that gave the probability of winning on election day, which included whatever their estimate was of uncertainty moving forward (including the probability of game-changing events, which is pretty low).

Re: Nassim Talebs case against Nate Silver is bad math

#117

Post by vertline3 » Tue, Apr 09, 2019, 9:30 PM UTC

I stopped paying attention when Silver said the Buckeyes only had a 2 percent chance to win the title, because I knew that was bogus, team was stacked. Then he did the same thing with the Cavs when they turned around and won, so it just seemed to fit what Taleb says about "predictors" in The Black Swan. Since then, I don't listen much to forecasters, their track record is dubious.

Re: Nassim Talebs case against Nate Silver is bad math

#118

Post by et2o » Tue, Apr 09, 2019, 9:31 PM UTC

ikeboy wrote: post #110

et2o wrote: post #104

Earlier quoted context omitted.

Curious about this. > It's how you can prove all non-Bayesian models are irrational Can you expand or link? > Error bars and spreads are already outside the Bayesian paradigm, which demands a single number Not following at all. What do you mean?

https://plato.stanford.edu/entries/epistemology-bayesian/sup... >Not following at all. What do you mean? One assumption is that given a bet, you must choose to take a side of it. If you won't bet for something at X%, you must bet against it at 1-X%.

Ah, interesting link. I think I ran into some confusion since 'rational' and 'Bayesian' are somewhat overloaded terms.

When I think of Bayesian analysis, I always think of Bayesian models which attempt to model uncertainty and always have a posterior distribution (spread) of outcomes.

For example, in the Dutch Book page, instead of having a point-prior (P=0.51 (delta on P=0.51?)), each degree of belief could have some (different) distribution which could yield a posterior with a distribution and change the math here.

Re: Nassim Talebs case against Nate Silver is bad math

#119

Post by trpc » Tue, Apr 09, 2019, 9:32 PM UTC

coldtea wrote: post #90

trpc wrote: post #59

I really want to be Nassim Taleb, I don't want to get a real job, I don't want to be a slave for any kind of rules or laws whether in corporates, government or academia, I am much bigger than that, I just want to make enough money trading and shorting stocks during an economic bubble and retire young and then set myself emperor of the world and make fun of whoever I don't like such those academics who can't make 6 fi…

Well, you're being sarcastic, but he does have a point in all that. The proof of the pudding being in the eating, and all that. There's something to be said for respecting somebody who betted on something he called and won (especially if they did it repeatedly), especially when most others were all against the very possibility of them being right... And being free with fuck you money does give you some freedom to say…

even though I am kinda sarcastic I extremely admire this kind of thinking (assuming that I got him right), many stock traders have this kind of free thinking mentality and disdain for all kinds of rules and authority, George Soros once said that the only reason people read about his philosophy is that he made a few billions, he knew from the very beginning that having enough money would make his ideas worth of being considered for smart people and absolutely right for retarded people. It's like money shifts the truth or something!

Re: Nassim Talebs case against Nate Silver is bad math

#120

Post by dmix » Tue, Apr 09, 2019, 9:32 PM UTC

iron0013 wrote: post #74

mathattack wrote: post #13

Taleb in person at a talk is very different than Taleb with a bottle of wine on Twitter. To both of their credit, they’ve expanded my knowledge of probability and statistics.

Have they though, or have they only made you feel as though your knowledge of probability and statistics has expanded? Taleb is the poster boy for "sound and fury, signifying nothing"

I don't know about Taleb, but Nate Silver's book on Bayesian statistics and prediction in general got me really interested in statistics from a general-reader standpoint. Where I ended up digging harder into the more math/sciencey books afterwards.

The part about predicting Earthquakes and his work on sports is really good.

https://www.amazon.com/Signal-Noise-Many-Predictions-Fail/dp...