PS: If you want to build or train your own model or make predications, you can find open (structured) data about all world cups at the football.db, see https://github.com/openfootball/world-cup and https://github.com/openfootball/world-cup.json Enjoy the beautiful game.
Goldman Sachs model to predict World Cup game results didn’t come close
11–20 of 136 posts
Re: Goldman Sachs model to predict World Cup game results didn’t come close
#12The World Cup is about the worst sporting event for data led predictions like this, far too much can rely on a few events that are basically a coin flip. It would be interesting to see how the predictions went for something like the Premiere League tables.
> far too much can rely on a few events that are basically a coin flip Can you give us some examples?
For example: Croatia got to the final via two penalty shootouts, which is very much like coin flipping.
Re: Goldman Sachs model to predict World Cup game results didn’t come close
#13The World Cup is about the worst sporting event for data led predictions like this, far too much can rely on a few events that are basically a coin flip. It would be interesting to see how the predictions went for something like the Premiere League tables.
I'd like to see some scientific evidence of this (i.e. using multiple experiments, null hypothesis, etc.)
I think that you have to come at this from an analytical direction rather than an empirical one; the problem is that everyone can say that the approaches that you have used to show how difficult it is to model the world cup are just the wrong ones and you need to redo the experiment.
Analytically the difference between premier league and the world cup is that you have momentum and continuity in the premier league and the world cup is essentially one shot. So in the PL team A will play team E and G and H before it plays team B, team B may play team E and H and Q (which played G). Team A may be winning games that your strength model shows they should lose, Team B may be losing games... and so on and so on. There is more evidence that might matter. More importantly you can be wrong quite a lot of the time in a season and still be right at the end of it (as the bounces of the ball even out over time). Not so much in the world cup - one goal knocks you out and there is no coming back! Basically the world cup demands an algorithm that works with less evidence and with a much higher degree of accuracy.
Re: Goldman Sachs model to predict World Cup game results didn’t come close
#14If anything, this is a clear illustration of poor use of probabilistic prediction. When used for investments you have many outcomes. If the model is any good, you will most of them right. In the World Cup you have very few. Even if you count all games played. Definitely not excusing Goldman Sachs here, they should have known better than to try to predict this. There was only a tiny chance this could be great advertis…
> they should have known better than to try to predict this.
There's no downside, only free publicity. If they, by good fortune and a following wind, get it right - then the publicity is incredible. If it's wrong they laugh and say "well, better stick to predicting what we're good at!" and they still get a shitload of headlines and awareness of their product.
This was not a mistake.
Re: Goldman Sachs model to predict World Cup game results didn’t come close
#15"Of course, past data don’t always predict the future; Goldman Sachs never tells clients to make decisions solely on the basis of its models’ findings"
in the real world Goldman Sachs would manipulate the games to make it looks like they got it right and their clients got richer.
Re: Goldman Sachs model to predict World Cup game results didn’t come close
#16The "Ludic Fallacy" strikes again [0].
> The ludic fallacy, identified by Nassim Nicholas Taleb in his 2007 book The Black Swan, is "the misuse of games to model real-life situations."
...
> The alleged fallacy is a central argument in the book and a rebuttal of the predictive mathematical models used to predict the future – as well as an attack on the idea of applying naïve and simplified statistical models in complex domains. According to Taleb, statistics is applicable only in some domains, for instance casinos in which the odds are visible and defined.
Both Taleb's books, "The Black Swan" and "Fooled by Randomness" are an interesting take for such models. Meanwhile, most economists know about "Knightian Uncertainty" [1] which talks about differentiation of risk and uncertainty.
> "Uncertainty must be taken in a sense radically distinct from the familiar notion of Risk, from which it has never been properly separated.... The essential fact is that 'risk' means in some cases a quantity susceptible of measurement, while at other times it is something distinctly not of this character; and there are far-reaching and crucial differences in the bearings of the phenomena depending on which of the two is really present and operating.... It will appear that a measurable uncertainty, or 'risk' proper, as we shall use the term, is so far different from an unmeasurable one that it is not in effect an uncertainty at all."
Re: Goldman Sachs model to predict World Cup game results didn’t come close
#17I watched quite a few matches and among the things I saw in the matches but not in any statistics are: - motivation (Germany and Croatia were the two extremes here, no idea how to measure it) - team cohesion (number of articles in a few journals questioning the team cohesion, maybe also articles about individual players) - creativity in offense (maybe measurable via "target missed from close distance" + "ball passed…
These kinda show what makes predicting football particularly difficult. I like the ideas, and I think we (or more likely some ML algorithm) can come up with the set of conditions that showed why France prevailed against the specific opposition at this specific World Cup ... but I suspect that the conditions would be pretty unique and invalid for Euro 2020, WC 2022 etc.
As you identified, motivation could be pretty hard to measure ... but even if we could it might be a pretty poor predictor anyway. France in the early stages didn't look very motivated, while England and Colombia looked pretty lively.
Team cohesion - the German team were pretty consistent (not dazzling, but consistent) and we know how that ended. Again France didn't really impress until the latter stages of the WC.
Creativity in offense - I guess it can indicate a sort of calm or confidence in front of goal but actually it can actually be seen as pretty negative. For example Arsenal a few years back came under fire for having plenty of possession in the 18 yard box but failing to convert. Spain's confident quick pass-and-move "tiki-taka" was ever-present and has in my eyes been impotent in the last few years (and more important as a neutral viewer - very frustrating to watch).
Defensive errors that didn't lead to a goal could be a nice indicator of the ability of a defence to pick up after each others mistakes - but at the same time these errors that lead to goals (i.e. Croatia's second goal in the final) are relatively rare and a lack of a goal could just point to the opposing team's inability to convert due to a poorly organised or a lack of opportunism from their strikers.
I'm not sure what you mean with the last one, but I think this could be a nice one - if you mean "times you lost possession in your own half". A profligate midfield and defence is bound to ship goals, I doubt there are many teams that can either fight back after trailing by a goal or two or score enough to maintain a reasonable buffer.
I applaud the effort though - it takes more creativity and care to think of some new angles (like you did) than to think of some possible counter examples (like I did)!
Re: Goldman Sachs model to predict World Cup game results didn’t come close
#18Leonid Bershidsky and a lot of other journalists laughing at Goldman Sachs' incorrect predictions seem to miss the point.
The World Cup predictions from Goldman Sachs (and also UBS) are a form of recreation and entertainment with machine learning. It's an expression of quant nerd humor.
Analogous intellectual games would be engineers devising ridiculous Rube Goldberg contraptions[1] or programmers building "enterprise" FizzBuzz[2].
(I think it would add to the fun if GS uploaded their raw data and models to Github for others to play with.)
>It certainly didn't predict the final opposing France and Croatia on Sunday.
True, but it did predict France having better chance winning overall but was handicapped by a tougher draw. It also predicted France beating Croatia in round 16 instead of the final. The pdf says:
>While Germany is more likely to get to the final, France has a marginally higher overall chance of winning the tournament,
[1] https://en.wikipedia.org/wiki/Rube_Goldberg_Machine_Contest#...
[2] https://github.com/EnterpriseQualityCoding/FizzBuzzEnterpris...
Re: Goldman Sachs model to predict World Cup game results didn’t come close
#19If anything, this is a clear illustration of poor use of probabilistic prediction. When used for investments you have many outcomes. If the model is any good, you will most of them right. In the World Cup you have very few. Even if you count all games played. Definitely not excusing Goldman Sachs here, they should have known better than to try to predict this. There was only a tiny chance this could be great advertis…
> they should have known better than to try to predict this. There's no downside, only free publicity. If they, by good fortune and a following wind, get it right - then the publicity is incredible. If it's wrong they laugh and say "well, better stick to predicting what we're good at!" and they still get a shitload of headlines and awareness of their product . This was not a mistake.
Well, there's the downside of articles like this pointing out they've had 4 years to work on their models and they've got worse
Re: Goldman Sachs model to predict World Cup game results didn’t come close
#20The World Cup is about the worst sporting event for data led predictions like this, far too much can rely on a few events that are basically a coin flip. It would be interesting to see how the predictions went for something like the Premiere League tables.
Premiere League is over a far too long period of time with variables that can change completely without prediction (managers getting sacked, players leaving / joining, etc).
The reasons events like the World Cup are far more interesting is because it's over a shorter period of time.
I think the problem here isn't the event but rather the sport. Something like snooker or tennis will offer the same brevity over the period but with chance playing a less significant role due to the number of games played per match.
That all said, if my years of watching snooker has taught me anything, it's that people are not machines and thus will perform vastly different from day to day depending on how what mood they're in.