Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The unreasonable difficulty of time series forecasting

suzyahyah.github.io

Post reply on HN
Print view View original

51–60 of 69 posts

Re: The unreasonable difficulty of time series forecasting

#51

Post by newsicanuse »

crystal_revenge wrote: post #49

> I’ve been thinking recently about what makes time series forecasting problems so difficult compared to other sequence learning tasks Whenever I teach people time series forecasting, I always point out that one of the biggest challenges is that you will always have values at prediction time that are out side the range of values observed during training (specifically the value of t ). In plenty of other machine learn…

To ensure that your forecaat don't fall outside the range of training dataset you can use some tree based model like Random Forest for time series forecasting which averages out the values.

Re: The unreasonable difficulty of time series forecasting

#52

Post by 10000truths »

crystal_revenge wrote: post #49

> I’ve been thinking recently about what makes time series forecasting problems so difficult compared to other sequence learning tasks Whenever I teach people time series forecasting, I always point out that one of the biggest challenges is that you will always have values at prediction time that are out side the range of values observed during training (specifically the value of t ). In plenty of other machine learn…

Shouldn't the training data be preprocessed to transform time values to a relative coordinate system? I would expect a time-series training pipeline to contain something like:

1. Define {N = context duration, M = forecast duration} upfront

2. Select some time value T

3. Extract historical data whose timestamps lie in time interval (T, T+N+M)

4. Transform timestamp values to (-N, M) interval by subtracting T+N from each timestamp

5. Append timestamp-transformed data to training data

6. Goto 2

Or are you saying that people don't want to define N and M upfront?

Re: The unreasonable difficulty of time series forecasting

#56

Post by bawis »

crystal_revenge wrote: post #49

> I’ve been thinking recently about what makes time series forecasting problems so difficult compared to other sequence learning tasks Whenever I teach people time series forecasting, I always point out that one of the biggest challenges is that you will always have values at prediction time that are out side the range of values observed during training (specifically the value of t ). In plenty of other machine learn…

> Whenever I teach people time series forecasting, I always point out that one of the biggest challenges is that you will always have values at prediction time that are out side the range of values observed during training (specifically the value of t).

I don't get this, time is usually not a covariate in ts models, so why is it a challenge?

Re: The unreasonable difficulty of time series forecasting

#57

Post by jamilton »

a-dub wrote: post #15

> Given the lack of forecasting signal, the obvious next step then is to seek out external (exogenous) features in the real world that can help prediction models. this seems right to me. maybe another interesting approach would be a fusion llm+ts model that does multiple-input-single-output with input metadata and causality narrative. so it "thinks" about what data it has and how predictive it may be of the target va…

There's a couple "AI forecaster" companies now, Preseen and FutureSearch. My understanding is you give them a query and they research every possible relevant factor and give an estimate of the likelihood of your query, and supposedly they're comparable to if you asked a human to do the same thing, but they're new enough that they're not really proven yet.

Re: The unreasonable difficulty of time series forecasting

#59

Post by a-dub »

jamilton wrote: post #57
a-dub wrote: post #15

> Given the lack of forecasting signal, the obvious next step then is to seek out external (exogenous) features in the real world that can help prediction models. this seems right to me. maybe another interesting approach would be a fusion llm+ts model that does multiple-input-single-output with input metadata and causality narrative. so it "thinks" about what data it has and how predictive it may be of the target va…

There's a couple "AI forecaster" companies now, Preseen and FutureSearch. My understanding is you give them a query and they research every possible relevant factor and give an estimate of the likelihood of your query, and supposedly they're comparable to if you asked a human to do the same thing, but they're new enough that they're not really proven yet.

something like that would be interesting. i just get the sense that doing it metadata blind would result in something that learns a somewhat brittle short-time covariance-like structure that doesn't use all available information to achieve best possible generalization and that in the end the benchmark numbers start to reflect some luck with that for predicting non-stationary behavior combined with a lot coming from a pretty good synthesizer that matches previous patterns.

that being the case, it seems the next jump in performance would come from incorporating both metadata and metadata enriched causality and maybe that next jump in performance would be the most interesting jump from a practical system that is useful perspective. (it's more valuable for a system to predict outlier events than it is for it to do an excellent job at synthesizing ordinary behavior)

Re: The unreasonable difficulty of time series forecasting

#60

Post by ekorkuda »

dcl wrote: post #48

I have to explain this to managers, execs and stakeholders all the time. ML models work great for systems where the rules/dynamics do not change over time. With Forecasting, in a lot of domains where you want a forecast, everything is subject to change - laws, policies, regs, customers appetites, competitors behaviours, etc.

I highly relate to this. But I always had a hard time to explain this knowledge that is in an engaging and non-jargon way. How do you do it usually? do you have a script/slides/conversation transcript (can be just your vague memory of it) you can share to help me on this
Post reply on HN