Why we should care about the Nate Silver vs. Nassim Taleb Twitter war • Hacker News Board

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#121

Post by ubernostrum » Tue, Dec 18, 2018, 12:41 PM UTC

zimablue wrote: post #87

To try and explain Taleb's argument as simply as possible: If someone tells you today that an event is 100% certain, and tomorrow tells you that it is impossible, then you intuitively will not trust their forecasting. If it's certain, then it's certain not to be impossible tomorrow Now if someone tells you something is a 90% chance today, and a 10% chance tomorrow, again you don't trust their predictions. There is a…

I've always treated 538's forecasts as meaning nothing more or less than "We have a model, we fed the current data into it and ran N simulations of the event, and in X% of those simulations, this was the result". Which, as I understand it, is literally how they're generated.

Any discussion of the accuracy of the model (which should be the only thing that admits debate) has to be retrospective. Just saying "well the forecasts changed" doesn't inherently discredit the model, especially because things like elections turn out to be highly sensitive to tiny variations (even something as simple as a rainy election day in a few key precincts can completely flip an outcome).

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#122

Post by DougBTX » Tue, Dec 18, 2018, 12:41 PM UTC

logifail wrote: post #93

_0ffh wrote: post #85

Earlier quoted context omitted.

Yeah, essentially "extremely possible" is just "far from impossible". Something can be very possible but still not very likely, surely?

If you say that X is extremely possible does that mean that X is unlikely to happen (probably won't), or that X is very likely to happen (probably will)? I'd never come across anyone using this phrase until Nate Silver did. Is it a US English thing?

I’ve not heard “extremely possible” used like that before, but it is pretty similar to “completely possible” or “entirely possible”, example in a headline: https://www.spectator.co.uk/2018/10/its-entirely-possible-to...

It doesn’t mean that a broken heart will definitely kill you, just that there is definitely a chance that it will kill you.

I could totally imagine Nate trying to emphasise the idea that 20% is a much bigger probability than say 1% by using language like that, but put too much weight on it here leading to confusion.

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#123

Post by OscarCunningham » Tue, Dec 18, 2018, 12:42 PM UTC

dmurray wrote: post #118

OscarCunningham wrote: post #101

Earlier quoted context omitted.

I think that Taleb's problem is that he thinks all the uncertainty in Silver's model comes from the fact that people might change their minds about who to vote for. This is why Taleb can't make a model that fits Silver's forecasts. Silver could only be confident early on if he knew that people weren't going to change their minds much. But if people don't change their minds much then Silver's forecast shouldn't fluctu…

No, it's mostly caused by people changing their minds on who to vote for (or whether to vote). That's why the estimates move around a lot when there is news affecting the election, like the given example of Comey re-opening the Clinton investigation.

There's still also a lot of uncertainty caused by polling error. You can tell by comparing the final polls to the actual results.

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#124

Post by Rainymood » Tue, Dec 18, 2018, 12:46 PM UTC

mturmon wrote: post #25

This is an intellectually confused post. It is so unclear and muddled that only a charitable reader would suppose the author understands the mathematical and modeling issues here. I’m surprised the author published it in this form - you don’t want to jump into a controversy like this with such an unclear discussion. As just one example, the whole digression on a “decision boundary” is conceptually mistaken. Once you…

I kind of feel the same. I literally shouted out "what the fuck is this shit" mid-way reading the post. Then I looked up the author's credentials: Ph.D. Candidate at Stanford | Chief Data Scientist at MatrixDS ... I'm not sure what to make of this.

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#125

Post by felipeerias » Tue, Dec 18, 2018, 1:02 PM UTC

safgasCVS wrote: post #94

mturmon wrote: post #25

This is an intellectually confused post. It is so unclear and muddled that only a charitable reader would suppose the author understands the mathematical and modeling issues here. I’m surprised the author published it in this form - you don’t want to jump into a controversy like this with such an unclear discussion. As just one example, the whole digression on a “decision boundary” is conceptually mistaken. Once you…

Unlikely. Taleb seems to like how his argument has been summarised. The crux of the issue taleb has with silver is showing probabilities without making a decision/declaration is cowardly. When asked about making a prediction he says one thing then follows it up with a “ but don’t be surprised if it could be completely random.” And that’s the point - the fact he covers his ass is the problem. Election forecasting is h…

IMHO what Nate Silver does is not a prediction on the outcome of future events, such that when he doesn't get it right it is because he has failed, but rather an analysis of existing data in order to provide a detailed description of our own uncertainty. And by seeing how this uncertainty evolves as more data comes in or different events happen, we get a better understanding of the underlaying process.

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#126

Post by jimnotgym » Tue, Dec 18, 2018, 1:07 PM UTC

All of which increases the probability that I will ignore Taleb's tweets. I follow people who are smart enough to disagree in a civil manner, and smart enough to recognise that there may be things one doesn't know.

The way he talked to renowned classicist (and feminist, which seemed to be a problem for Taleb) Dame Mary Beard in another spat was disgusting. Trying to impose statistical certainty on ancient history ended up making him look pretty dumb to everyone but him.

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#127

Post by lixtra » Tue, Dec 18, 2018, 1:11 PM UTC

mhb wrote: post #119

dagw wrote: post #115

Earlier quoted context omitted.

But that doesn't seem to be the nature of their disagreement. Taleb's main complaint seems to be about how the swings in the prediction probabilities far out from the event indicates a fundamental flaw in how the underlying model uses probability. That being said it would be a fun little exercise to see who much money would be made if they bet had $1 on every race 538 predicted in 2016 and 2018 using the model odds.

It doesn't need to be a simple bet. It could incorporate the volatility which Taleb might believe shouldn't be present in an unflawed model.

Indeed after skimming through the paper[1] (I find it hard to grasp without a solid background) it seems that Taleb wants to play this game every day before the election and expects to come out ahead on election day when all the bets are settled.

[1] https://arxiv.org/pdf/1703.06351.pdf

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#128

Post by kilburn » Tue, Dec 18, 2018, 1:16 PM UTC

ummonk wrote: post #72

VintageCool wrote: post #55

Earlier quoted context omitted.

This "Twitter fight" began with Nate Silver talking to the media, trying to convey both his prediction and its uncertainty: https://thehill.com/homenews/sunday-talk-shows/414759-nate-s... > "So in the House we have Democrats with about a 4 in 5 chance of winning," Silver told ABC's "This Week." > "But no one should be surprised if they only win 19 seats and no one should be surprised if they win 51 seats," Silver add…

I normally like Taleb's insights but it was obvious when following this that he (and Dinesh D'Souza) were ridiculously misconstruing Nate Silver's comment. Two outcomes being "extremely possible" in no way means they have a 50/50 chance of happening. It merely means that neither outcome has a low enough chance that you should assume it won't happen.

Using the phrase "extremely possible" as an expert is either a gross mistake or intentional misguidance.

"Possibility" is like "optimality" in the sense that they are binary attributes and thus don't admin grades. Something either is or isn't possible. Qualifying something as "more possible" is a mistake, and qualifying it as "extremely possible" is just nonsense.

A guy who lives from probabilistic analysis should now better.

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#129

Post by CosmicShadow » Tue, Dec 18, 2018, 1:18 PM UTC

Animats wrote: post #65

pmart123 wrote: post #33

I liked the first chapter of "Fooled by Randomness," but I stopped reading the book when each chapter regurgitated the same points, each more nihilistic than the last. Perhaps Taleb should concentrate on the accuracy of his own predictions: https://www.businessinsider.com/taleb-every-single-human-bei... >>> "every single human being" should bet U.S. Treasury bonds will decline. >>> Short the S&P vs Long Gold, in a 5…

I'd be more impressed with Taleb if he published the results for his Emperica fund for years other than 2008. Taleb's strategy was to buy options way out of the money. In boring years, the fund lost money. It did really well in 2008. Not enough numbers have been published to determine if his strategy was a net win over a business cycle. The win seems to have coming from the fund's short lifetime including 2008.

You only need to win once. One of his big things is about optionality and anti-fragility. If you buy something that barely hurts you in terms of cost, but can sky rocket massively if something big happens, you can capitalize on it, and all other costs were meaningless. Even VCs try for that, although with numbers that aren't nearly as good.

What do you mean by a business cycle? If his fund just goes until it makes it big and then he stops, is that not a business cycle? Did he continue? I don't know. It just seems weird to ask about a business cycle in the case of a strategy that is very long played, waiting for unpredictable results. The only thing that matters is can you play long enough to hit that result if that is your game? It seems clear that it did win, or else he'd probably be focusing more on it until he did? It's not like a regular business with actual output or production that can be easily replicated and measured, it's just a play till you win or lose deal and I'm sure there are others that did similar things and lost, just like many more companies are dead than alive and we only see the alive ones.

I understand wanting to see the numbers, but this is just one sample for a strategy.

Re: Why we should care about the Nate Silver vs. Nassim Taleb Twitter war

#130

Post by killjoywashere » Tue, Dec 18, 2018, 1:47 PM UTC

hn_throwaway_99 wrote: post #37

mturmon wrote: post #25

This is an intellectually confused post. It is so unclear and muddled that only a charitable reader would suppose the author understands the mathematical and modeling issues here. I’m surprised the author published it in this form - you don’t want to jump into a controversy like this with such an unclear discussion. As just one example, the whole digression on a “decision boundary” is conceptually mistaken. Once you…

Agreed. The more I read the more I got embarrassed for the author, mainly because his points were so ridiculously flimsy. I thought this was one of the worst: > Because FiveThirtyEight only predicts probabilities, they do not ever take an absolute stand on an outcome: No ‘skin in the game’ as Taleb would say. This is not, however, something their readers follow suit on. In the public eye, they (FiveThirtyEight) are j…

> basking in self-proclaimed genius when luck happened to run in his favor.

I have read read Black Swan and I think Taleb's central thesis is there are valid epistimic gaps in every model, and that suggest there is at least one family of hedge strategies where the goal is to think as creatively as possible about what we don't know, but what could be, so as to reframe that lack of knowledge as a plan to go tromping around in a jungle looking for holes. But to do so requires wide-ranging knowledge across many domains, which is hard to develop.