Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The forecasting fallacy (2020)

alexmurrell.co.uk

Post reply on HN
Print view View original

61–70 of 71 posts

Re: The forecasting fallacy (2020)

#61

Post by cesaref »

cnewey wrote: post #25
altdataseller wrote: post #24

Earlier quoted context omitted.

Both of those examples have been exploited to death and no longer are profitable

If the parent had discovered a viable and profitable trading strategy, do you think they would share it here?

I've been out of the loop for a number of years, but I believe there's a serious amount of ML being thrown at predicting time series, so this probably gives you an idea of how money is being made.

I've no idea what the current models look like. I imagine it was all RNNs but maybe transformers have taken over?

Re: The forecasting fallacy (2020)

#62

Post by lordnacho »

The critical distinction is IMO that the examples he gives are all examples of where the prediction forms part of its own outcome. Whether interest rate go up depends on whether people think interest rates will go up. Same with GDP and all the other examples.

Having this recursion is a massive problem, because there might not be a stable point where the prediction and its outcome are fixed.

This may be why it's hard to steer cars automatically. I don't know enough about the subject, but it would seem to fit in the category of "fields where what you think affects what happens".

By contrast, a lot of other phenomena are complicated but do not have this prediction-feedback effect. The weather, astronomical observations, a lot of engineering systems.

Re: The forecasting fallacy (2020)

#63

Post by kromem »

civilized wrote: post #21

There's a germ of insight here that could use some development and nuancing. > The future is uncertain. You cannot predict it. But you can create it. For millions of years, prediction has been the engine by which humans have created the future. We don't always call it that, but prediction is the engine. Let's start from the most basic facts of life. We know from experience (our own and others') what plants will susta…

While true, it's quite an underappreciated factor and rarely considered seriously by people.

You are correct that the human ability to manifest our imagination is remarkable.

We dream of flying like a bird, eventually we build machines to do it, and then it's taken for granted as part of our society.

In the 2nd century Lucian thinks up impossible things, comes up with a ship of men flying up to the moon, and less than two millennia later the impossible is increasingly old history.

Often, our invention even outpaces imagination. I can remember watching GATTACA long before watching its primary plot point become obsolete with CRISPR.

But despite a longstanding trend of imagination and prediction preceding invention, I regularly see those things dismissed as fantasy when they haven't happened yet. Much like the op-ed saying humans wouldn't fly with a million more years of effort just a year shy of the Wright brothers, I often see a disbelief in human advancement over the status quo as the more predominant perspective that the opposite, especially as the imagined thing is more fantastical.

So as an example, I've been talking privately a lot since GPT-3 about how digital resurrection is going to be an increasing trend to the point nearly everyone alive and online today will have a digital twin that outlives them. But because it's considered weird and "Black Mirror" in the present it's often dismissed as fantasy, even though the foundational concept of resurrection is at least as old as the Sumerians and there's already a patent granted to a trillion dollar company for an early technical vision of it. The Wright brothers' plane wasn't going to change the world as it existed that year, but the improvements over the future did.

It's seeing the trend of incremental improvements to early versions of a thing that I think people struggle with, and moreso now than a generation before as we've become so conditioned to shorter windows of consideration. "AI can't do my taxes only three years after allegedly doing impossible things that alarmed the most cited scientists? It must be a fad like crypto."

Long term vision of sufficiently advanced technology looks more and more like magic, and we're used to dismissing magic as fantasy. So when the things we imagined or predicted in fantasy begin to take shape with emerging technology, we ignore the past trends of realizing fantasy with technology and overemphasize the present state and limitations in predicting the future of it.

Re: The forecasting fallacy (2020)

#64

Post by kromem »

cesaref wrote: post #16

If you want a counter example, go and investigate algo trading hedge funds - you'll find they do a pretty solid job of predicting the future. Sure, some of them predict only a few ms into the future, some a few minutes (the one I worked for was in that category) and others will do interday strategies. I'm pretty sure there are examples which have a track record of decent returns above the markets they trade in with l…

At this point algo trading is a race to exploit the other algo created patterns, and is less predicting inherent market inefficiencies as much as fellow algo trading inefficiencies.

Self-fulfilling prophecy is a thing, even when the prophecy is algorithmic.

Re: The forecasting fallacy (2020)

#65

Post by roenxi »

kqr wrote: post #40
diab0lic wrote: post #37

Much of the discussion here including the linked article fail to make an important distinction between domains. Prediction can be done quite effectively on thin tailed processes. A lot of the counter examples listed in the comments here are physical systems which are thin tailed. I see aircraft autopilot, collision detection, ozone depletion. These are all well understood physical phenomenon in which large deviations…

It's not that we can't forecast heavy tailed processes -- it's just that the forecasts are used wrong. The appropriate layman's forecast of a recession within the next year is something like a constant 11 %. I'm willing to bet this outperforms most "predictions" out there. But! When people see that number they go, "right, so it's vastly more likely it does not happen" and then completely ignore the possibility. The p…

Thankyou for putting it eloquently. The other subtle problem is that people tend to ask silly questions. Maybe I've just been lucky so far, but the focus on recessions seems like a relic of the pre-1970s time when GDP correlated with wages. I'm not seeing why it matters so much these days.

Interest rates are centrally planned, so it is a bit of a philosophical question to ask what it means to "forecast" them. Really it is more a question of politics than any reliable process and the answer is "lower than makes sense". The economy for the last 10 or 20 years looked like it is carrying a lot of value-destructive companies looking for an opportunity to collapse that were almost certainly related to cheap credit. Uber and WeWork spring to mind. US debt is accumulating in a way that would have been making people nervous for decades.

The problem here isn't forecasting, it is money being corralled to purposes that consume more resources than get generated. Being unable to predict what the perturbation against trend will be in 12 months time doesn't seem useful, appears to be impossible and it is weird that people love to focus on it. There are much more serious things to keep an eye on.

Re: The forecasting fallacy (2020)

#66

Post by jmalicki »

diab0lic wrote: post #37

Much of the discussion here including the linked article fail to make an important distinction between domains. Prediction can be done quite effectively on thin tailed processes. A lot of the counter examples listed in the comments here are physical systems which are thin tailed. I see aircraft autopilot, collision detection, ozone depletion. These are all well understood physical phenomenon in which large deviations…

That you're talking about both collision detection and economics is very telling for the issues a lot of people have with things like these.

When an economist predicts a heightened probability of a recession, people believe the economist, and adjust their investments to avoid it, this is seen as a failure in prediction.

Yet when a collision detection system detects a collision, and the car brakes to avoid the collision, this is also a false positive as no collision happened!

It's not clear why people see these as different. When the system takes the prediction into account to avoid the predicted calamity, we shouldn't see that as a failure in prediction.

Re: The forecasting fallacy (2020)

#67

Post by jakderrida »

hardlianotion wrote: post #8

Everyone that provides a forecast that others depend on should really be on the hook to report on the outcome, and to provide the forecast error distribution, if the forecast is one they make regularly.

I remember actually importing all the WSJ economic forecast surveys and testing the accuracy of each forecaster to find out which one I should bet my money on in the future. The results were... unimpressive for all of them. What they call forecasts aren't really rigorously modeled predictions of the future. Pretty much all of them come down to a very simple formula of long-term mean reverting growth rates. The forecasters that got awards for best forecast were either always optimistic and got the award during good economic periods or always pessimistic and got it during an unprecedented recession. So disappointing. I really believed in it before I studied the numbers.
Post reply on HN