The forecasting fallacy (2020)
alexmurrell.co.uk
The forecasting fallacy (2020)
1–10 of 71 posts
Re: The forecasting fallacy (2020)
#2I like the article but not the final conclusion "We can’t predict much at all", I think we can predict more than we think, it is similar to asking if the glass is half empty and half full.
For example, sci-fi, scientists, and some economists have predicted a lot of things before but we don't have an accurate time of happening: one thing is to predict an event for next year and another see a trend that will happen in the next 50 years. There is even a futurist science.
Regarding AI, and forgetting "AI cults" it is incredible that the neural networks that we are using now are similar to the ones studied decades ago but there was a breakthrough in other aspects such as computing capacity and techniques [1].
Re: The forecasting fallacy (2020)
#3What?!
The article ends with a Alan Kay quote attributed to Cindy Gallup:
>Or as Cindy Gallup likes to say:
>
>“In order to predict the future, you have to invent it".
So she does like to say it (see quote below), but it seemed strange to end with a "second hand" quote.
From https://www.lbbonline.com/news/5-minutes-with-cindy-gallop
LBB > ‘In order to predict the future, you have to invent it’, Alan Kay, is reportedly your favourite quote. Why?
CG > Because I am all about inventing the future. Too many people feel that the future is something that happens without us, that we have no control over, that simply rolls us over in its wake. I believe in deciding what you want the future to be, and then inventing it.
Re: The forecasting fallacy (2020)
#4Language is important. We can predict anything and everything. There are some things we can't _reliably_ predict.
This article ironically suffers from its own thesis. It assumes that because we haven't provided some things successfully in the past, we will never predict anything in the future.
A simple counterexample should dispel this silly notion. We used to consider the weather completely unpredictable. Now we have elaborate systems and theories that allow us to predict the weather with at least some accuracy.
A more reasonable thesis might be, we can't reliably predict human behavior, because much like the uncertainty principle, each prediction that is published, which it must be to be meaningful, affects the behavior it is trying to predict.
Re: The forecasting fallacy (2020)
#5You can predict what will occur quite easily, you can't necessarily predict when it will occur.
A lot of the "failed" predictions relate to markets...the reason why you can't predict this stuff is because humans are irrational and those irrational humans control outcomes in the short and medium term.
For example, you can see that a recession should have occurred. What people didn't expect is fiscal stimulus worth about 50% of GDP, tens of trillions in monetary stimulus, etc. Yes, if the government just deposits hundreds of billions into people's bank accounts then it is going to impact growth.
I remember back in 2007, Blackstone RE made insane leveraged bets at the very top of the market, it is very easy to point out rationally "these are absolutely terrible investments, the price is awful, these aren't economic"...today, all these bets got bailed out by the government (after a short period of bankruptcy/restructuring), the person responsible is probably going to be made head of Blackstone, that unit has hundreds of billions in AUM, etc.
The assumption that people make with forecasts has to be: the long-term is today. That is it. You will often be wrong but that does not mean that your model is wrong (indeed, the reason why this stuff is so predictable is because people believe that the models have stopped working repeatedly).
If you take something as apparently "unpredictable" as the market, you can predict returns to within 10bps very easily over the long-term because the fundamentals do not change (but, again, the current period has been the most unpredictable because of the level of government intervention, it is unprecedented...the government cannot hold back the waves forever though).
EDIT: referencing the 2005 interest rate prediction is quite humorous too, 2% against a predicted 5%...this was basically the start of it. Back then, no-one thought the Fed would cut rates to this level for, essentially, no reason...the Fed cut, the result was a financial crash. Turns out those predictions (which were essentially the long-term neutral rate) were right and the Fed was wrong...but the only account you hear about is: those damn forecasters, they failed to predict the Fed torching the economy, so stupid. Lol.
Re: The forecasting fallacy (2020)
#6I think this article has two shortcomings that make its sweeping conclusions shaky.
First, it identifies forecasting with point forecasting. There are other ways to put forecasting questions, e.g. lower and upper level with a certain probability.
Also it mentions Tetlock, but only his negative findings, not his positive ones that lead to Good Judgement Project, which suggest the contrary of the conclusion of this article [1].
Thus I think it is not up to the latest research results.
See you over at gjopen.com, if you are interested and have lots of time to waste...
[1] https://en.m.wikipedia.org/wiki/The_Good_Judgment_Project
Re: The forecasting fallacy (2020)
#7I think this article has two shortcomings that make its sweeping conclusions shaky. First, it identifies forecasting with point forecasting. There are other ways to put forecasting questions, e.g. lower and upper level with a certain probability. Also it mentions Tetlock, but only his negative findings, not his positive ones that lead to Good Judgement Project, which suggest the contrary of the conclusion of this art…
Tetlock paints a different picture of his positive findings than you do.
Specifically, Tetlock's project opens with key issues of scope about what to even try to forecast. Based on his previous work in expert prediction, he concluded that geopolitics is sufficiently chaotic to be impossible to predict 10 years out. So while he did a lot of work on forecasting, it is generally focused on the next year or to.
Which means that Tetlock agrees that we can't predict 10 years out.
Re: The forecasting fallacy (2020)
#8Everyone that provides a forecast that others depend on should really be on the hook to report on the outcome, and to provide the forecast error distribution, if the forecast is one they make regularly.
Re: The forecasting fallacy (2020)
#9Any forecasts that don't involve probabilities or confidence intervals are useless. Also, if any forecasters were really serious, they would register their past forecasts and show how good they have been in the past. But I think most of forecasters are probably afraid of showing their true track record.
Re: The forecasting fallacy (2020)
#10One should ask economists what a recession is, not how to predict one. Good modelers do not necessarily need (or want) to know what they are predicting and still beat "domain experts".
Authority without clear track-record is a net negative to getting good results. It is better to stick to anonymity, and only let the track-record do the talking/weighting. Without a clear track-record it does not even matter if the prediction-maker has skin in the game. If you do have skin in the game, there is no reason to sell your hide cheaply, or even give it away. You instead take the profit others say does and can not exist beyond "luck": If you can't even beat a random walk, you have no business evaluating the limitations of predictive modeling.
The big consultancy companies making bold predictions don't even need to be right. Customers read the predictions these consultancy companies peddle, because these customers are not bold enough to make their own predictions. And nobody ever got fired for buying the predictions from big consultancy companies and incorporating them into a business strategy.