Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The unreasonable difficulty of time series forecasting

suzyahyah.github.io

Post reply on HN
Print view View original

31–40 of 69 posts

Re: The unreasonable difficulty of time series forecasting

#31

Post by pishpash »

evo wrote: post #28
EdwardAF-IT wrote: post #16

A very interesting article, suzyahyah. I especially appreciated your definition of stationarity, a concept with which I struggled in my own time series class. If I understand correctly, it sounds like the basic premise is that a fundamentally statistical methodology (LLMs) can't realistically predict a non-stationary data generation, which makes sense. Separately, I've wondered for some time if there might be some re…

I think there's a close linkage between "predictable" and "stationary". Ultimately, if you strip everything away, either there's a core where the future looks like the past (which is equivalent to being stationary), or there isn't. That core could be "the laws of physics and base conditions are stationary", and everything else is deterministic functions applied on top, but the fundamental process is trying to find th…

That's a bit of a vacuous statement. Stationarity makes sample statistics meaningful due to the LLN, sure, but almost nothing is stationary (and even if something is, there is no way to know anyway, you can only assume). If you condition on enough variables and do enough transformations you may get something seemingly stationary and therefore trivially predictable. But all the practical complexity is in that structure you need to specify. The true prediction "problem", when people work on such "problems", really is in the stuff besides the thing that you can just use averages to predict.

Re: The unreasonable difficulty of time series forecasting

#32

Post by gofreddygo »

c7b wrote: post #23

Well. Lots of math that boils down to 'predicting the future is hard'. Especially when the future is one of social construction, that's what gets lost a bit here. Predicting the future is easier for planetary motions than for Bitcoin.

The unreasonable difficulty of predicting (actual) future effects based on past effects with no regards to their cause.

Re: The unreasonable difficulty of time series forecasting

#33

Post by pishpash »

jldugger wrote: post #18
klodolph wrote: post #4

I see that a lot of these are markets. Yes, it’s hard to predict markets. Because anybody who can successfully predict markets, does so, makes money, and changes the market so their predictions lose their edge. Time series forecasts are a lot easier if you are forecasting, say, disk use in your servers or whatnot. (By “easy” I mean you can do a simple prediction and get useful insights.)

It's true that markets are more _adversarial_. But there's still a lot of trouble with distribution shifts even in server metrics. As an example, our SRE team got paged a few times in the past month for traffic drops due to the World Cup. This stresses the nowcasting alert in several dimensions: - there's no seasonal pattern to the matches, they happen sorta randomly. - they drive increased query traffic in the hour…

Unless disconnected from the world, or very thinly coupled, most processes become hard to predict in a generalizable way for this reason.

Re: The unreasonable difficulty of time series forecasting

#34

Post by pishpash »

vjk800 wrote: post #22

Again and again nerds coming from maths/CS/etc to finance are surprised to find that financial time series are actually impossible to predict. After spending almost a decade in finance now with a similar background, the arrogance of the "let's just throw in some neural network/whatever and be done with it" attitude now amuses me. After all, if it was easy - or even possible with any kind of effort - to predict (even…

Which naturally begs the question whether the excess returns in finance (outside of services provided for liquidity matching, risk transformation, etc.) is not just exploiting inside information of one kind or another. If you have inside information, your time series forecasts are going to be excellent and no simulation is needed.

Re: The unreasonable difficulty of time series forecasting

#35

Post by joe_the_user »

I remember in 2017, it was said that "regular" machine learning techniques were better for time series than neural networks. It seems like this blog post is indirectly references that belief and gives. It seems like the idea is simpler ML can be tuned to a given time series whereas neural network require lots of training.

Also, one method works better than another is more meaningful than "predicting the future is hard".

Re: The unreasonable difficulty of time series forecasting

#36

Post by pishpash »

hintymad wrote: post #13

The examples in the posts suggest that the past does not contain all the patterns, or information in general, about the future. If so, isn't it natural that point forecast will fail in some cases?

The past cannot contain all information about the future, or there is some superdeterminism going on, by definition.

Re: The unreasonable difficulty of time series forecasting

#37

Post by kjellsbells »

jldugger wrote: post #18
klodolph wrote: post #4

I see that a lot of these are markets. Yes, it’s hard to predict markets. Because anybody who can successfully predict markets, does so, makes money, and changes the market so their predictions lose their edge. Time series forecasts are a lot easier if you are forecasting, say, disk use in your servers or whatnot. (By “easy” I mean you can do a simple prediction and get useful insights.)

It's true that markets are more _adversarial_. But there's still a lot of trouble with distribution shifts even in server metrics. As an example, our SRE team got paged a few times in the past month for traffic drops due to the World Cup. This stresses the nowcasting alert in several dimensions: - there's no seasonal pattern to the matches, they happen sorta randomly. - they drive increased query traffic in the hour…

The UK power grid operators famously plan for massive demand surges at eg the end of major football matches. I cant imagine what their forecasters thought of the England-Mexico nail biter. Half the nation heading to put the kettle on, half glued to their seats. Would love to see the charts of demand now that the World Cup is done...

Re: The unreasonable difficulty of time series forecasting

#38

Post by kenjackson »

c7b wrote: post #23

Well. Lots of math that boils down to 'predicting the future is hard'. Especially when the future is one of social construction, that's what gets lost a bit here. Predicting the future is easier for planetary motions than for Bitcoin.

Yes. Or even to go a step further -- once we exclude all the things that aren't hard to predict, we're left with the things that are.

Re: The unreasonable difficulty of time series forecasting

#39

Post by jbritton »

EdwardAF-IT wrote: post #16

A very interesting article, suzyahyah. I especially appreciated your definition of stationarity, a concept with which I struggled in my own time series class. If I understand correctly, it sounds like the basic premise is that a fundamentally statistical methodology (LLMs) can't realistically predict a non-stationary data generation, which makes sense. Separately, I've wondered for some time if there might be some re…

This brings up the philosophical question of whether humans have free will or is everything deterministic following the laws of physics.
Post reply on HN