Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The unreasonable difficulty of time series forecasting

suzyahyah.github.io

Post reply on HN
Print view View original

21–30 of 69 posts

Re: The unreasonable difficulty of time series forecasting

#21

Post by mr_toad »

giyanani wrote: post #17

> Given the lack of forecasting signal, the obvious next step then is to seek out external (exogenous) features in the real world that can help prediction models. After all many real world time series are event-driven (FX, bitcoin) and hence they are exposed to shocks and drifts which can also be measured or accounted for in the data generating process. This is the approach I'm using at my job, which is incident dete…

> We can then add in an existing dataset of _known_ incidents, indexed by the same common features, as a training/validation set.

Beware of the Anna Karenina principle. Well behaved data might be explicable by the same common features, but often the anomalies all have unique characteristics.

https://en.wikipedia.org/wiki/Anna_Karenina_principle

Re: The unreasonable difficulty of time series forecasting

#22

Post by vjk800 »

Again and again nerds coming from maths/CS/etc to finance are surprised to find that financial time series are actually impossible to predict. After spending almost a decade in finance now with a similar background, the arrogance of the "let's just throw in some neural network/whatever and be done with it" attitude now amuses me. After all, if it was easy - or even possible with any kind of effort - to predict (even within some error margins or probability) what the stock prices are tomorrow, anyone doing it would quickly become a billionaire or trillionaire. If the person could keep doing it, at some point they would own enough of the stock market so that their actions would affect the prices and whatever pattern they found would vanish.

Financial markets are not a natural phenomenon that exist unchanged regardless of whoever is observing them. Their dynamics continuously change in response to collective actions of all of the humanity.

Oil price changed quite a bit when the US attacked Iran. If you are trying to predict the price of oil, your model would have to be able to predict Trump ordering an attack on Iran. Does your model include a full simulation of the mind of the president of the United States (and every other person who have any kind of impact on the world events)? If not, then your time series forecasts are not going to be that great.

Re: The unreasonable difficulty of time series forecasting

#24

Post by [deleted] »

gregw2 wrote: post #19
klodolph wrote: post #4

I see that a lot of these are markets. Yes, it’s hard to predict markets. Because anybody who can successfully predict markets, does so, makes money, and changes the market so their predictions lose their edge. Time series forecasts are a lot easier if you are forecasting, say, disk use in your servers or whatnot. (By “easy” I mean you can do a simple prediction and get useful insights.)

Between the difficulty and value propositions for forecasting timeseries "disk usage" versus "financial markets" there are some rather relevant time series such as "company sales" or "demand for company product" that come up again, and again, and again but are neither "easy" nor "predicting-the-stock-market-hard".

[deleted]

Re: The unreasonable difficulty of time series forecasting

#26

Post by joe_the_user »

mjburgess wrote: post #8
joe_the_user wrote: post #7

The question "what probability distribution generated this" is very hard to answer if you're only getting short window before the distribution changes. A lot of distributions could generate relatively "short" set dispersed data points and deciding which distribution is "really" creating the points might not even be meaningful. A distribution is a mathematical device, not something with a physical existence.

Probability distributions don't generate data, they describe our uncertainty about its generation retrospectively. So there is no answer to that question. The lack of such an answer is at the heart of why such inference is hard: in most cases, there isnt a stable underlying reality of anything doing the distributing.

I think we're saying roughly the same thing. It's natural starting with a statistical model to say it generates data (as the article sometimes does) but in reality, you often/generated have data generated by a possibly deterministic and completely different process but satisfying (if you're lucky) the statistical conditions. Even in testing random/stochastic models, you would usually use pseudo-random generators - which are deterministic.

Re: The unreasonable difficulty of time series forecasting

#27

Post by prerok »

EdwardAF-IT wrote: post #16

A very interesting article, suzyahyah. I especially appreciated your definition of stationarity, a concept with which I struggled in my own time series class. If I understand correctly, it sounds like the basic premise is that a fundamentally statistical methodology (LLMs) can't realistically predict a non-stationary data generation, which makes sense. Separately, I've wondered for some time if there might be some re…

The weather is an example of a non-linear dynamic system. Try as we might, we still cannot predict it really even a few days in advance. The stock market is much worse, even, so, no, it cannot be predicted.

Re: The unreasonable difficulty of time series forecasting

#28

Post by evo »

EdwardAF-IT wrote: post #16

A very interesting article, suzyahyah. I especially appreciated your definition of stationarity, a concept with which I struggled in my own time series class. If I understand correctly, it sounds like the basic premise is that a fundamentally statistical methodology (LLMs) can't realistically predict a non-stationary data generation, which makes sense. Separately, I've wondered for some time if there might be some re…

I think there's a close linkage between "predictable" and "stationary". Ultimately, if you strip everything away, either there's a core where the future looks like the past (which is equivalent to being stationary), or there isn't. That core could be "the laws of physics and base conditions are stationary", and everything else is deterministic functions applied on top, but the fundamental process is trying to find that stationary core.

In the finance space, the stationary core is often some 'stylized fact' that you're hypothesizing will hold true. This could be e.g., the momentum factor, that if you strip away the noise, there's an underlying trend that will hold over an extended duration.

Re: The unreasonable difficulty of time series forecasting

#30

Post by chas »

EdwardAF-IT wrote: post #16

A very interesting article, suzyahyah. I especially appreciated your definition of stationarity, a concept with which I struggled in my own time series class. If I understand correctly, it sounds like the basic premise is that a fundamentally statistical methodology (LLMs) can't realistically predict a non-stationary data generation, which makes sense. Separately, I've wondered for some time if there might be some re…

Chaos theory does provide us with a tool here: The Lyapunov exponent[1] and the related Lyapunov time[2]. These allow you to characterize how far into the future you can expect to be able to predict the behavior of a dynamical system. To Prerok’s example of predicting the weather, this is why 3 day weather forecasts tend to be great, 10 day weather forecasts tend to be good, and weather forecasts two months out are just the general average trends for the climate in that time and place.

There are chemical systems where Lyapunov time is small enough that you can only predict seconds or minutes into the future and astronomical systems that are nonlinear and chaotic but have a long enough Lyapunov time that you can make reasonable predictions for millions of years. For both of these scales, the Lyapunov time still bounds how far into the future you can expect your predictions to remain near to the actual behavior of the system.

I do not know the Lyapunov times of the financial markets. That said, mathematically-sophisticated professional analysis frequently get their predictions wrong in major ways, so I expect there is a pretty hard bound on predictive quality caused by a short Lyanpunov time of the markets themselves.

[1] https://en.wikipedia.org/wiki/Lyapunov_exponent

[2] https://en.wikipedia.org/wiki/Lyapunov_time

Post reply on HN