Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nassim Talebs case against Nate Silver is bad math

m.nautil.us

Post reply on HN
Print view View original

181–190 of 213 posts

Re: Nassim Talebs case against Nate Silver is bad math

#181

Post by groupdeterminac »

aubreyclayton wrote: post #103

So... I wrote this. Maybe I can clarify a little what I meant by the statement "Since Silver’s forecasts begin with probability models, it’s safe to assume they obey all the rules, including Bayes’, and would be arbitrage-free.", since this seems to be confusing some people: What I'm addressing here is Taleb's claim that Silver's probabilities would allow for arbitrage (i.e., riskless profit, not just profit on avera…

Needing help with one part of this. Say in the simplest case of a coin flip and two instants in time I assess probabilities:

  t=0: P(heads) = 0.75, P(tails) = 0.25
  t=1: P(heads) = 0.60, P(tails) = 0.40
These are consistent (or behave as probabilities or what have you) at each time (separately), so de Finetti's argument covers each (separately). Does this alone somehow protect from arbitrage over time? Or is the martingale stuff in the next paragraph, though postured as only a technical rephrasing, essential? Unsure since that part's over my head.

The above's enough to pose the question, but continuing for concreteness: if a buyer can determine any "significant" pattern in my assessments over time--for instance maybe my past assessments have been routinely seen to tend to a uniform distribution over time--aren't I still vulnerable to arbitrage, or else what's protecting me?

Re: Nassim Talebs case against Nate Silver is bad math

#182

Post by tweezy »

Someone1234 wrote: post #177
tweezy wrote: post #172

Earlier quoted context omitted.

Hey I don't have a dog in this fight, but I think this is the link you're looking for. https://github.com/fivethirtyeight/checking-our-work-data

Nope. I'm looking for 538's predictions. You cannot go from the raw data they used back to how they interpreted it.

Please look at the actual data on the linked page. Each of the csv files on that page has thousands of predictions 538 made going back as far as 2008. This isn't the raw data they used to make the predictions, these are the actual predications that they made.

Edit: For example, here are the headers on the file named presidential_elections.csv

year,office,state,district,election_date,forecast_date,forecast_type,party,candidate,projected_voteshare,actual_voteshare,probwin,probwin_outcome

These are the actual forecasts they made for presidential elections going all the way back to 2008.

Re: Nassim Talebs case against Nate Silver is bad math

#183

Post by wilg »

Someone1234 wrote: post #177
tweezy wrote: post #172

Earlier quoted context omitted.

Hey I don't have a dog in this fight, but I think this is the link you're looking for. https://github.com/fivethirtyeight/checking-our-work-data

Nope. I'm looking for 538's predictions. You cannot go from the raw data they used back to how they interpreted it.

Here is the historical data for the 2018 House forecast:

- Web: https://projects.fivethirtyeight.com/2018-midterm-election-f...

- JSON: https://projects.fivethirtyeight.com/2018-midterm-election-f...

You are correct they do not publish the exact math of the forecasts, but you can see how the forecast changed over time.

Re: Nassim Talebs case against Nate Silver is bad math

#184

Post by tweezy »

wilg wrote: post #183
Someone1234 wrote: post #177

Earlier quoted context omitted.

Nope. I'm looking for 538's predictions. You cannot go from the raw data they used back to how they interpreted it.

Here is the historical data for the 2018 House forecast: - Web: https://projects.fivethirtyeight.com/2018-midterm-election-f... - JSON: https://projects.fivethirtyeight.com/2018-midterm-election-f... You are correct they do not publish the exact math of the forecasts, but you can see how the forecast changed over time.

The link I provided does actually show every single forecast 538 has ever made. OP was wrong when they said it was the raw data they use to make the predictions. Check out the link I provided, it has everything OP was asking for.

Re: Nassim Talebs case against Nate Silver is bad math

#185

Post by niyikiza »

fineline wrote: post #155
oarabbus_ wrote: post #137

Earlier quoted context omitted.

Taleb is simultaneously both worth reading as well as one of the most pretentious, unpleasant asses I've ever read. The Black Swan was mostly fantastic, but Antifragile was mostly regurgitation. The biggest problem with him is he thinks he's much, much smarter than he is, like a kind of inverse-Dunning-Kruger effect. It's odd because he is quite a smart man in the first place; there's no need for him to be such a pre…

I've never read Black Swan. I found Antifragile hugely thought-provoking. Knowing both, would you imagine I had anything to gain from also reading Black Swan, or do you really think they basically cover the same ground?

I read both, and Fooled By Randomness. I would say you stand to gain a lot reading about the older ones as well.

Re: Nassim Talebs case against Nate Silver is bad math

#186

Post by mathattack »

iron0013 wrote: post #74
mathattack wrote: post #13

Taleb in person at a talk is very different than Taleb with a bottle of wine on Twitter. To both of their credit, they’ve expanded my knowledge of probability and statistics.

Have they though, or have they only made you feel as though your knowledge of probability and statistics has expanded? Taleb is the poster boy for "sound and fury, signifying nothing"

Taleb expanded my view of the impact of fat tails. (And my interpretation of his book helped me in an interview) He also helped me realize the limitation of models.

Re: Nassim Talebs case against Nate Silver is bad math

#187

Post by Izkata »

credit_guy wrote: post #179
bambax wrote: post #142

Earlier quoted context omitted.

I have a hard time understanding this: what does it mean to assign probabilities to one-off events?? The only way to verify probabilities predictions is to test it a great number of times. Otherwise you can say anything and never be wrong (except for 0 or 100%).

Every event is a one-off event. In particular the Trump election was the most one-off event in recent memory :) Kidding aside, your observation is correct, you need to perform repeated predictions. 538 does just that. They keep predicting a whole lot of outcomes, so you can check their track record. They even have a challenge for the audience, where you can record your own predictions and compare them with their corr…

> In the case of the November 2016 election, other prediction sites were giving the Trump team a 1-2% chance, 538 was giving them a 20% change. Considering that, 538 comes out as the clear winner.

One common mistake I've seen is people mistaking that 20% as the expected vote ratio. It definitely wasn't; that was hidden behind an apparently too-well-hidden toggle (buttons above the graph), and was floating around 45% Trump and 48% Clinton, with overlapping error ranges.

Re: Nassim Talebs case against Nate Silver is bad math

#188

Post by maps »

oska wrote: post #139
andrewla wrote: post #14

As I understand it, the feud in its current form started from a tweet from Dinesh D'Souza [1] in which he interpreted Nate Silver's statement that "Dems, GOP winning House are 'both extremely possible'" to mean that they had gone from an 80% chance of winning to a 50% chance. Taleb inexplicably chose this as the hill to die on [2] and wrote a lengthy and technical paper to refute, and replied (on the [2] thread) "Whe…

> But in this case I'm having trouble following what his criticism even is A piece by Isaac Faber [1] from December last year clearly described what Taleb's position is. A summary paragraph towards the end: > Here we can say, with some confidence that FiveThirtyEight predictions are not always reliable probabilities. However, they masquerade as one, being between 0 and 1 and all. This is Taleb’s primary argument; Fiv…

By far a much better write up of the issue.

Re: Nassim Talebs case against Nate Silver is bad math

#189

Post by iainmerrick »

dragonwriter wrote: post #147
iainmerrick wrote: post #60

Earlier quoted context omitted.

Huh, yeah, that sounds very plausible. A forecast that really did try to be an option pricing model like that would be interesting to see. It would have the advantage over a “now cast” that you could actually run the numbers and see how accurate it is. Whereas nobody can ever know what would actually happen if there were an election now rather than a year from now.

> A forecast that really did try to be an option pricing model like that would be interesting to see. Silver’s election forecast models are exactly that. The nowcast expressly is not, but it's also not the headline model. > It would have the advantage over a “now cast” that you could actually run the numbers and see how accurate it is. That's true, and not just in theory: Silver has recently run the numbers for all o…

Aha, thanks for clarifying that. I actually read that article, but the earlier comment got me confused over whether it was merely validating the final forecasts, or all forecasts over time.

Makes sense that it was written as an explicit rebuttal to Taleb -- but without mentioning him directly. :)

Re: Nassim Talebs case against Nate Silver is bad math

#190

Post by lolc »

ikeboy wrote: post #149
lolc wrote: post #146

Earlier quoted context omitted.

A price set by coin-flips does not revert to any mean. It's a random walk and it diverges. The idea that heads and tails must eventually even out is a common fallacy.

You don't need the price to revert, you just need to have a better estimate of the underlying probability.

Oh so the coin-flipper is not setting the price, but trying to predict it? I think at least I understand what you're explaining. I don't see how that is a useful model for what is going on though.

Predicting the result of a vote for a day when it's not going to be held is a form of entertainment that can't be verified. Is Silver claiming any more than that?

Post reply on HN