Live data from Hacker News

Goldman Sachs model to predict World Cup game results didn’t come close

bloomberg.com

41–50 of 136 posts

Re: Goldman Sachs model to predict World Cup game results didn’t come close

#41
post #31
post #23

People love to beat up on these companies because of this stupid world cup prediction. Yes, Goldman is a giant vampire squid wrapped around the face of humanity (Matt Taibi quote). But it turns out it's really just great marketing for their research teams. Also, I've seen some people say (not in this forum) that banks now look stupid because they're in the business of making predictions and they can't even get the wo…

I am pretty sure Banks make money on predictions if they get people on following them.

Yes... This is less a prediction and more a legal form of front running.

Re: Goldman Sachs model to predict World Cup game results didn’t come close

#42
post #18

Leonid Bershidsky and a lot of other journalists laughing at Goldman Sachs' incorrect predictions seem to miss the point. The World Cup predictions from Goldman Sachs (and also UBS) are a form of recreation and entertainment with machine learning. It's an expression of quant nerd humor. Analogous intellectual games would be engineers devising ridiculous Rube Goldberg contraptions[1] or programmers building "enterpris…

On the other hand, this is a predictive task that has defined outcomes and clear historical data - by my understanding, it is easier than commercial uses of machine learning [at least, easier to measure the effectiveness]. It's also Goldman Sachs and UBS choosing to attach their names to these and stake some reputation on these predictions. If they had hit the bullseye, they would be lauding these results.

Yup, the worst thing is that if they had got it right it would have been more or less due to pure chance, and it would have led to business flowing their way!

Re: Goldman Sachs model to predict World Cup game results didn’t come close

#43
post #20

The World Cup is about the worst sporting event for data led predictions like this, far too much can rely on a few events that are basically a coin flip. It would be interesting to see how the predictions went for something like the Premiere League tables.

Premiere League is over a far too long period of time with variables that can change completely without prediction (managers getting sacked, players leaving / joining, etc). The reasons events like the World Cup are far more interesting is because it's over a shorter period of time. I think the problem here isn't the event but rather the sport. Something like snooker or tennis will offer the same brevity over the per…

Interesting observations here about team sports have a bunch of extra opportunities for randomness. Do you know if there's anything equivalent to Fargo Rate for snooker (or other individual sports like tennis)?

Re: Goldman Sachs model to predict World Cup game results didn’t come close

#44
post #35

Duh. Looks like there's a fundamental misunderstanding of how statistics works all around. The probability of an event does NOT predict a particular outcome. Ever. It only says that if the experiment is performed again and again and again, like a few thousand times, then X% of those will match that probability. If I toss a fair coin you cannot predict the next outcome. You can only say that if I toss the coin a 1000…

I agree completely with your opening remarks.

>"You can only say that if I toss the coin a 1000 times, then close to 500 are going to turn up heads, and another 500 are going to turn up tails."

Sometimes you can do that and every single flip will be heads. It's unlikely, and across zillions of universes you'd only find it once - but we don't have a pool of universes that we can sample statistically.

Re: Goldman Sachs model to predict World Cup game results didn’t come close

#46
post #41
post #31

Earlier quoted context omitted.

I am pretty sure Banks make money on predictions if they get people on following them.

Yes... This is less a prediction and more a legal form of front running.

In what way is this a legal form of front running?

Re: Goldman Sachs model to predict World Cup game results didn’t come close

#47
post #23

People love to beat up on these companies because of this stupid world cup prediction. Yes, Goldman is a giant vampire squid wrapped around the face of humanity (Matt Taibi quote). But it turns out it's really just great marketing for their research teams. Also, I've seen some people say (not in this forum) that banks now look stupid because they're in the business of making predictions and they can't even get the wo…

The line between marketing making and prop trading is blurrier than you think. Whenever you quote a price you're implicitly making a prediction on the future of the market.

Re: Goldman Sachs model to predict World Cup game results didn’t come close

#49
post #30

> And in any case, the model only generated probabilities of winning a game and advancing, and no team was given more than an 18.5 percent chance of winning the World Cup. > [...] > But Goldman Sach’s misfire is perhaps the most curious. The model said, that there is a lot of uncertainty, and as it happens, it was entirely correct. A World Cup chance of 18.5 percent means, that 4 out of 5 times the team will not win,…

But Goldman Sachs are the kings of predicting uncertainty! This is their whole business! They make billions predicting certainty through the murky, uncertain waters of the global economy. Would you argue that the global economy is more uncertain that soccer? I'd say so. How is it that they can find success in the market but not in soccer?

I think this is a smoke signal. Soccer is corrupt; you can't predict the winner unless you know what's being passed around under the table. Goldman Sachs does these predictions so people read between the lines to see how corrupt it is.

My argument is: "Goldman is amazing at statistical analysis and they routinely practice it on much tougher models (the global economy), so they should have no problem predicting a simpler model (soccer). But since they drastically failed at predicting soccer, then there must be an equally drastic variable missing from their predictions. Since we can trust Goldman to use all available public information in their analysis, there must be critical information that is hidden from the public which affects the outcomes". I make some assumptions, but it's fairly sound, no?

Re: Goldman Sachs model to predict World Cup game results didn’t come close

#50

The "Ludic Fallacy" strikes again [0]. > The ludic fallacy, identified by Nassim Nicholas Taleb in his 2007 book The Black Swan, is "the misuse of games to model real-life situations." ... > The alleged fallacy is a central argument in the book and a rebuttal of the predictive mathematical models used to predict the future – as well as an attack on the idea of applying naïve and simplified statistical models in compl…

Damn, do I disliked Nassim Taleb. I don't think I've ever heard him say anything deep. That wikipedia article is an excellent.

In [0] you have the following:

> The ludic fallacy, identified by Nassim Nicholas Taleb in his 2007 book The Black Swan, is "the misuse of games to model real-life situations."

And he gives an example of this:

> One example given in the book is the following thought experiment. Two people are involved:

> Dr. John who is regarded as a man of science and logical thinking

> Fat Tony who is regarded as a man who lives by his wits

> A third party asks them to "assume that a coin is fair, i.e., has an equal probability of coming up heads or tails when flipped. I flip it ninety-nine times and get heads each time. What are the odds of my getting tails on my next throw?"

> Dr. John says that the odds are not affected by the previous outcomes so the odds must still be 50:50.

> Fat Tony says that the odds of the coin coming up heads 99 times in a row are so low that the initial assumption that the coin had a 50:50 chance of coming up heads is most likely incorrect. "The coin gotta be loaded. It can't be a fair game."

> The ludic fallacy here is to assume that in real life the rules from the purely hypothetical model (where Dr. John is correct) apply. Would a reasonable person bet on black on a roulette table that has come up red 99 times in a row (especially as the reward for a correct guess is so low when compared with the probable odds that the game is fixed)?

So Nassim Taleb wanted to discuss "using games to model real-life situations" and to demonstrate the pitfalls he uses two characters. He _portrays_ the characters as "man of logical thinking" vs "man who lives by his wits", but as we'll see he's missing one dimension to his characterization.

The first problem here is that implicitely he's suggesting to the reader that the decisions of the "man of logical thinking" represent the pitfalls of "applying games to model real-life situations", whereas the the other guy's decision represent.... it's not specified, but clearly has a better outcome.

The second problem, is that he conflates "applying something you read on some textbook to real life without thinking" with "modelling real-life". He suggests to the reader that those two people are actually "logical" vs "instinct", but they're not. They're a dumb guy who knows maths vs a smart guy who doesn't know math. _Obviously_ real-life is more complex than your textbook examples, and so the smart guy is going to win because his fuzzy heuristics beat the first guys decisions which are optimal within his flawed model. An actual smart and logical person would update his model based on new evidence (i.e. "I was told that this coin was 50-50 but actually the chance of what I just saw is so small that it's more likely that I was just lied to") and then use maths to make predictions and beat the guy who's smart but doesn't know math.

So ironically, he wants to portray the dangers of using over-simplified models and to do that he uses an example where he obscured one dimension.

Nassim Taleb is really good a rhetoric but light on substance.

[0] https://en.wikipedia.org/wiki/Ludic_fallacy

Post reply on HN