Live data from Hacker News

Nassim Talebs case against Nate Silver is bad math

m.nautil.us

201–210 of 213 posts

Re: Nassim Talebs case against Nate Silver is bad math

#201
post #37
post #27

Earlier quoted context omitted.

The best criticism I can come up with is that 538 probably doesn't do a sufficient job of explaining to its audience that their published probabilities are predictions for what would happen if the event occurred today . This seems to be one of Taleb's (and others') criticisms of their predictions: that they don't consider the probability of major unexpected events occurring between now and the event (like an athlete…

That seems like a pretty big difference to gloss over. Why are they publishing probabilities for if the event were to happen today? The event isn't happening today!

There are two types of people consuming these projections. I'll call them the sportsbook type and the newspaper reader. The sportsbook consumers want to know what the probabilities are for the actual event, when it occurs (election day). The newspaper consumers want to know what is going on currently and the country's political temperature.

538's forecasts, especially far from election day, are for the newspaper consumer. They provide insight into current political dynamics, and a quantitative look at how people view policy, the economy, where current fundraising is at, etc. in a single concise number that can serve as the basis for further discussion and analysis. They however aren't useful for the sportsbook consumer, because they don't account for the practically innumerable "unknown unknowns" that will occur between now and election day (scandals, terrorist attacks, fluctuations in the economy, flubs on the campaign trail, external activist efforts gaining traction). If they did account for all that uncertainty, then the predictions consistently be near 50-50 for the majority of the race, which not only is still uninteresting for the sportsbook types, but also means you no longer are providing value to the newspaper types.

Its important to point out that, over time, the predictions become applicable to both readers. As election day nears, the number of "unknown unknowns" decreases, and a prediction for "what would happen if an election were held today" and "what will happen on election day" converges. So up until the few months and weeks leading up to election day, the predictions should be taken more as commentary, and then perceived as serious indications of what will occur closer to election day.

Re: Nassim Talebs case against Nate Silver is bad math

#202

Earlier quoted context omitted.

"[2] become less variable over the remaining time to expiration ... And it's just not true, as can be seen even in a simple random-walk model." Dear @aubreyclayton, I'm genuinely interested in seeing how you arrived at this conclusion. Would you kindly share the proof, or at least explain the logic behind it. Thank you.

Sure. If you've studied stochastic processes like Brownian motion, check out the derivation starting with the last paragraph on p.4 of this note: https://drive.google.com/file/d/1tQj4ZGja6jKADJGiFj-dcQAirWo... If that's foreign to you, you might be interested in this write-up I did about election-forecasting in which I considered the same example, just in discrete time rather than continuous time: http://nautil.us/is…

Thank you.

Re: Nassim Talebs case against Nate Silver is bad math

#203

Dropping in to say that Nassim is literally a genius. Him being rude doesn't change that - it just makes you a postmodernist.

"Postmodernist"? Are you just lobbing out random words that you think will be perceived as insults? Go gather your thoughts--if you are capable of doing so--and try again.

Re: Nassim Talebs case against Nate Silver is bad math

#204

Earlier quoted context omitted.

Taleb is simultaneously both worth reading as well as one of the most pretentious, unpleasant asses I've ever read. The Black Swan was mostly fantastic, but Antifragile was mostly regurgitation. The biggest problem with him is he thinks he's much, much smarter than he is, like a kind of inverse-Dunning-Kruger effect. It's odd because he is quite a smart man in the first place; there's no need for him to be such a pre…

I've never read Black Swan. I found Antifragile hugely thought-provoking. Knowing both, would you imagine I had anything to gain from also reading Black Swan, or do you really think they basically cover the same ground?

There is a lot of overlap in the Venn Diagram between the two. I would say read the first 80-100 pages of Black Swan and decide if you would like to complete the book.

Re: Nassim Talebs case against Nate Silver is bad math

#205
post #56
post #23

Earlier quoted context omitted.

> I don't think there's a constraint on "Nate Silver has a probability model" that mathematically guarantees "Nate Silver's probability model" is correct If I'm understanding the article correctly, it's not talking about models being "correct," it's just talking about the probabilities of all the outcomes adding up to 100%, which means you can't place a bet that guarantees you make money regardless of the outcome. In…

If over time, your forecasts predictably change, then you can be arbitraged. If e.g. I know that you'll have a probability below 40% at some point in the future, and I know you'll have a number above 41% at some point in the future, then it's trivial to create a strategy over time that is guaranteed to make money. Taleb's claim, as I understand it, is that Silver is incorporating random noise into his model and so wi…

Silver's forecasts do predictably change, more precisely in that they swing deterministically in favour of one candidate whenever the most recent poll to come in favours that candidate heavily.

Each time the probability drops below 40% (in your example) it is likely because the fair odds swung in the latest polls. To attempt to take advantage of this is to ignore the recent evidence (conditional upon that evidence being quite different to the current concensus).

It's not at all unreasonable to do so, but isn't arbitrage. What if all polls started being increasingly conclusive leading up to election? 40, 39, ..., 0. No surprises and no arbitrage.

Perhaps another way to express this critique of Silver's methodology is that it lends too much weight on recent outlier observations? Sure if you knew this you could profit off these swings, but making the arbitrage argument doesn't add much to the discussion to my mind.

Re: Nassim Talebs case against Nate Silver is bad math

#206
There are several problems with both of them.

(1) Most of Nassim Taleb’s math is more branding and showing off than actual science or statistics. It has been stated by people who are far smarter than Taleb that “if you can’t state it simply then you don’t understand it.” If Richard Feynman can explain physics in a relatable way, then taleb can talk about statistics in the same. He doesn’t because explaining is not his objective, personal branding is.

Taleb uses obscure, nonsensical math to show off and brow beat opponents. In my experience, working with people who are actually smart, this is not what very educated and intelligent people do.

The general public may fall for pictures of pages of obscure nonsensical calculations depicting god knows what, all I see is fraudulent personal marketing.

I say this having worked for and known CEOs who use the same tricks to lie to investors. It’s bullshit.

(2) Nate Silver runs a media opinion network. I followed his work and early on they did very rigorous analysis and had great and accurate visualizations which did seem to accurate predict results.

For whatever reason, I saw five thirty eight evolve into a bunch of media talking points and opinion mongering. The coverage on five thirty eight for the 2016 election resembled strongly that of standard news networks.

Why? Because if you want to make money on advertising you need to completely fill the airwaves with as much content as possible and opinion content is the cheapest and easiest to produce.

My opinion is that Nate slipped. His data analysis got distracted by running a media business, which focuses on advertising revenue and being likable by target demographics.

Frankly I don’t trust either of them. I also side more strongly with Taleb - predicting outcomes with random hidden variables is an exercise in chicanery.

No one wants to hear that the universe is random and that predicting it is impossible for long stretches of time. Look at the comments here, 90% of people (probably programmers) think that if they can just find the right equations then predicting the future is possible.

Nope.

Re: Nassim Talebs case against Nate Silver is bad math

#207
Academic math: deriving precise solutions to predictive equations (eg @NateSilver538)

Risk taker math: evaluating exposure to potential distributions (eg @nntaleb)

With whom would you invest? Also, a useful heuristic to detect charlatans. Our knowledge of future never precise.

more at https://twitter.com/_NicT_/status/1115818015004688386

Re: Nassim Talebs case against Nate Silver is bad math

#209

Dropping in to say that Nassim is literally a genius. Him being rude doesn't change that - it just makes you a postmodernist.

"Postmodernist"? Are you just lobbing out random words that you think will be perceived as insults? Go gather your thoughts--if you are capable of doing so--and try again.

Postmodernists think that being upset about someone's personality changes the very fabric of spacetime. "If he's mean, he must be wrong." That's hardly the case.

Re: Nassim Talebs case against Nate Silver is bad math

#210

Earlier quoted context omitted.

"Postmodernist"? Are you just lobbing out random words that you think will be perceived as insults? Go gather your thoughts--if you are capable of doing so--and try again.

Postmodernists think that being upset about someone's personality changes the very fabric of spacetime. "If he's mean, he must be wrong." That's hardly the case.

That's not what "postmodernism" means. Care to try again?
Post reply on HN