Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nassim Talebs case against Nate Silver is bad math

m.nautil.us

Post reply on HN
Print view View original

161–170 of 213 posts

Re: Nassim Talebs case against Nate Silver is bad math

#161

Post by ummonk »

ikeboy wrote: post #56
baddox wrote: post #23

Earlier quoted context omitted.

> I don't think there's a constraint on "Nate Silver has a probability model" that mathematically guarantees "Nate Silver's probability model" is correct If I'm understanding the article correctly, it's not talking about models being "correct," it's just talking about the probabilities of all the outcomes adding up to 100%, which means you can't place a bet that guarantees you make money regardless of the outcome. In…

If over time, your forecasts predictably change, then you can be arbitraged. If e.g. I know that you'll have a probability below 40% at some point in the future, and I know you'll have a number above 41% at some point in the future, then it's trivial to create a strategy over time that is guaranteed to make money. Taleb's claim, as I understand it, is that Silver is incorporating random noise into his model and so wi…

That isn’t arbitrage. Arbitrage entails making a guaranteed profit, not relying on predictable fluctuations.

Re: Nassim Talebs case against Nate Silver is bad math

#162

Post by comex »

ikeboy wrote: post #129
Permit wrote: post #127

Earlier quoted context omitted.

> An arbitrage is just a strategy that's guaranteed to make money. If you know for sure that something will go up and down at different points you could make money risk free. Simply buy at any point below the current price and sell at any point above. This is not arbitrage. Also this is precisely the kind of strategy Taleb would have argued against in his book about black swan events.

Time arbitrage is arbitrage. If someone was flipping a coin and raising their price when it landed heads and lowering it when tails, arbitraging using reversion to the means would be a perfectly legitimate arb strategy. Taleb's argument is that Silver's estimates are based on random noise, and are thus vulnerable to this form of arbitrage.

It think it’s… sort of the reverse? Flipping a coin results in Brownian motion, which is an example of a “martingale”, a function whose expected future value is equal to its current value; and the evolution of a share price being a martingale (at least under a modified probability measure) is exactly the condition for the market to not have arbitrage:

https://en.m.wikipedia.org/wiki/Risk-neutral_measure

I’m not sure exactly how the arbitrage works, though.

But the argument in the paper seems to be not that there’s noise but that there’s too much of it. In fact, it assumes that the expected vote share over time is subject to Brownian motion with specific parameters (which is a rather flawed assumption), but then concludes that 538’s probability distribution is more swingy what he’d calculate from those parameters.

Re: Nassim Talebs case against Nate Silver is bad math

#163

Post by throwawaymath »

dictum wrote: post #131
throwawaymath wrote: post #46

Earlier quoted context omitted.

> If Silver really believed these probabilities were correct, he should be willing make bets with these odds, otherwise his incentives are distorted. Yes, the "skin in the game" argument. That is Nassim Taleb's calling card. Personally, I consider it a little unreasonable to expect people to literally put their money where their mouth is every time they make a forecast.

It's a kind of ad hoc censitary suffrage. One can only put possessions on the line if they have them, and the greater one's possessions, the greater their perceived "skin".

Yeah I get the idea, I just don't like that it encourages dismissal of opinions which aren't backed by some form of collateral.

Re: Nassim Talebs case against Nate Silver is bad math

#164

Post by comex »

ikeboy wrote: post #159
comex wrote: post #157

Earlier quoted context omitted.

Well, in theory you can divide the uncertainty into two parts, by supposing that the outcome is driven by a fundamentally stochastic process and you also have imperfect information about the parameters of that process. For example, the outcome could be determined by a coin flip but you’re not sure whether the coin is fair. In that case, there is a “true” probability of heads based on the nature of the coin, and separ…

Sure, but as a good Bayesian you should be willing to bet based on that expected value, not based on the spread. Re polls: still more complicated than that, even with 100% polling you'd still have response error and non-voters. Much of the difference between polls is their assumptions about demographics of voters.

Sure, but that doesn’t mean 538 can’t include information about the spread, in addition to their top-line number which is just a single probability.

And yes, my discussion of polling error was oversimplified, but I was mainly just trying to explain what I meant about having to treat something as random if it’s theoretically predictable given enough information.

EDIT: Hold on; I don’t think 538 does include error bars for their probabilities. They have error bars for measurables like the vote share, but that’s normal. So I don’t understand what this argument is even about :)

Re: Nassim Talebs case against Nate Silver is bad math

#165

Post by ikeboy »

ummonk wrote: post #161
ikeboy wrote: post #56

Earlier quoted context omitted.

If over time, your forecasts predictably change, then you can be arbitraged. If e.g. I know that you'll have a probability below 40% at some point in the future, and I know you'll have a number above 41% at some point in the future, then it's trivial to create a strategy over time that is guaranteed to make money. Taleb's claim, as I understand it, is that Silver is incorporating random noise into his model and so wi…

That isn’t arbitrage. Arbitrage entails making a guaranteed profit, not relying on predictable fluctuations.

Arbitrage over time is properly called arbitrage. If the fluctuations are known in advance, then you make a guaranteed profit by trading over time.

Re: Nassim Talebs case against Nate Silver is bad math

#166

Post by ikeboy »

comex wrote: post #162
ikeboy wrote: post #129

Earlier quoted context omitted.

Time arbitrage is arbitrage. If someone was flipping a coin and raising their price when it landed heads and lowering it when tails, arbitraging using reversion to the means would be a perfectly legitimate arb strategy. Taleb's argument is that Silver's estimates are based on random noise, and are thus vulnerable to this form of arbitrage.

It think it’s… sort of the reverse? Flipping a coin results in Brownian motion, which is an example of a “martingale”, a function whose expected future value is equal to its current value; and the evolution of a share price being a martingale (at least under a modified probability measure) is exactly the condition for the market to not have arbitrage: https://en.m.wikipedia.org/wiki/Risk-neutral_measure I’m not sure…

The point is that Silver's predictions are the result of coin flips, but that the underlying probability isn't. If he's updating further than he should then you can make money by updating the correct amount and betting against him.

Re: Nassim Talebs case against Nate Silver is bad math

#167

Post by ikeboy »

comex wrote: post #164
ikeboy wrote: post #159

Earlier quoted context omitted.

Sure, but as a good Bayesian you should be willing to bet based on that expected value, not based on the spread. Re polls: still more complicated than that, even with 100% polling you'd still have response error and non-voters. Much of the difference between polls is their assumptions about demographics of voters.

Sure, but that doesn’t mean 538 can’t include information about the spread, in addition to their top-line number which is just a single probability. And yes, my discussion of polling error was oversimplified, but I was mainly just trying to explain what I meant about having to treat something as random if it’s theoretically predictable given enough information. EDIT: Hold on; I don’t think 538 does include error bars…

Someone several comments up suggested Silver should include error bars and should bet based on a spread, I said that was not proper Bayesian methods, then we had this back and forth.

Re: Nassim Talebs case against Nate Silver is bad math

#168

Post by mike_ivanov »

aubreyclayton wrote: post #156
tomp wrote: post #47

Earlier quoted context omitted.

But is he wrong?

He is wrong in that the sentence of his that I quoted contained two statements: "when the volatility of the underlying security increases, [1] arbitrage pressures push the corresponding binary option to trade closer to 50% and [2] become less variable over the remaining time to expiration." His calculation proves [1], but it's [2] that is the basis of his criticism of Silver. And it's just not true, as can be seen ev…

Prove it.

Re: Nassim Talebs case against Nate Silver is bad math

#169

Post by Someone1234 »

KC8ZKF wrote: post #102
Someone1234 wrote: post #89

Earlier quoted context omitted.

It is worth noting that 538 doesn't keep a historical record of their predictions, making it difficult to really see how they predict outcomes coming up to a conclusion. You can only see the latest/final prediction. 538 also predicted lower chances of victory in the weeks and months leading up to that, then adjusted it on election day vanishing away their earlier guesses. This isn't wrong on the face of it, but peopl…

The 2016 election predictions are still posted. https://projects.fivethirtyeight.com/2016-election-forecast/...

Yes the newest one. Not the historical ones.

Re: Nassim Talebs case against Nate Silver is bad math

#170

Post by Someone1234 »

dragonwriter wrote: post #111
Someone1234 wrote: post #89

Earlier quoted context omitted.

It is worth noting that 538 doesn't keep a historical record of their predictions, making it difficult to really see how they predict outcomes coming up to a conclusion. You can only see the latest/final prediction. 538 also predicted lower chances of victory in the weeks and months leading up to that, then adjusted it on election day vanishing away their earlier guesses. This isn't wrong on the face of it, but peopl…

> It is worth noting that 538 doesn't keep a historical record of their predictions, making it difficult to really see how they predict outcomes coming up to a conclusion. You can only see the latest/final prediction. It's worth noting that you are completely wrong: not only do they keep such a record, they also provide a handy chart of it on the latest prediction page so you can easily see exactly how the prediction…

Where? Go ahead and link to it.
Post reply on HN