A minor language error: "this model does not outperform classical statistical methods neither in accuracy nor speed." should say "either" and "or".
Exponential Smoothing: faster and more accurate than NeuralProphet
11–20 of 44 posts
Re: Exponential Smoothing: faster and more accurate than NeuralProphet
#12As usual in ML, the appropriate solution depends on the problem and context. ML (particularly DL) tends to outperform "classical" statistical time series forecasting when the data is (strongly) nonlinear, highly dimensional and large. The opposite holds as well. It is also important to note that accuracy is not the only relevant metric in practical applications. Explainability is of particular interest in time series…
> ML (particularly DL) tends to outperform "classical" statistical time series forecasting when the data is (strongly) nonlinear, highly dimensional and large. This claim about forecasting with DL comes up a lot, but I’ve seen little evidence to back it up. Personally, I’ve never managed to have the same success others apparently have with DL time series forecasting.
It's true simply because large ANNs have a higher capacity, which is great for large, nonlinear data but less so for small datasets or simple functions.
In any case, Transformers are eating ML right now and I'm actually surprised there's no "GPT-3 for time series" yet. It's technically the same problem as language modeling (that is, multi-step prediction of numerics), however, there is only a comparably little amount of human-generated data for self-supervised learning of a time series forecasting model. Another reason might be that the expected applications and potentials of such a pre-trained model aren't as glamorous as generating language.
Re: Exponential Smoothing: faster and more accurate than NeuralProphet
#13This wouldn't pass peer-review of it were a paper. Major issues:
- No fair hyperparametrization for Neural Prophet. They mention multiple times they used default hyperparams or ad-hoc example hyperparams.
- 3/4 benchmark datasets (one they didn't finish training) where ETS outperforms is not strong evidence of all-round robustness. Benchmarks like SuperGlue for NLP combine 10 completely different tasks with more subtasks to assess language model performance. And even SuperGlue is not uncontroversial.
Re: Exponential Smoothing: faster and more accurate than NeuralProphet
#14Can someone explain this? I don't know what the context is for this Show HN.
They're time series prediction methods. E.g. they mention electricity usage forecasting - given historical data, what will the usage be in 1 hour?
Facebook's Prophet is quite popular in the space I understand. No idea about the other two.
Re: Exponential Smoothing: faster and more accurate than NeuralProphet
#15A minor language error: "this model does not outperform classical statistical methods neither in accuracy nor speed." should say "either" and "or".
The "Not-Neither-Nor" sequence is typical, even with regards to American English, versus British English (the Queen's English.) In either case, both are technically-correct.
Re: Exponential Smoothing: faster and more accurate than NeuralProphet
#16As usual in ML, the appropriate solution depends on the problem and context. ML (particularly DL) tends to outperform "classical" statistical time series forecasting when the data is (strongly) nonlinear, highly dimensional and large. The opposite holds as well. It is also important to note that accuracy is not the only relevant metric in practical applications. Explainability is of particular interest in time series…
A larger problem is that time series modeling is particularly resistant towards black box approaches since a lot of information is encoded in the model itself.
Take even a simple moving average model on daily observations. Consider stock ticker data (where there are no weekends) and web traffic data (where there is an observation each day). The stock ticker data should be smoothed with a 5 day window and the web traffic with a 7 to help reduce the impact of weekly effects (which probably shouldn't exist in the stock market anyway).
It's possible in either of these cases you might find a moving average that performs better on some choose metric, say 4 or 8 days. However neither of these alternatives make any sense as a window if we're trying to remove day-of-week effect, and unless you can come up with a justifiable explanation, smoothing over arbitrary windows should be avoided.
If you let a black box optimize even a simple moving average you would be avoiding some very essential introspection into what your model is actually claiming.
Not to mention that we often can do more than just prediction with these intentional model tunings (for example day-of-week effect can be explicitly differenced from the data to measure exactly how much sales should increase on a Saturday)
Re: Exponential Smoothing: faster and more accurate than NeuralProphet
#17This wouldn't pass peer-review of it were a paper. Major issues: - No fair hyperparametrization for Neural Prophet. They mention multiple times they used default hyperparams or ad-hoc example hyperparams. - 3/4 benchmark datasets (one they didn't finish training) where ETS outperforms is not strong evidence of all-round robustness. Benchmarks like SuperGlue for NLP combine 10 completely different tasks with more subt…
While the results don't prove the superiority convincingly, it does seem that ETS is a good candidate as a first go-to in practical applications. "In practice, practice and theory are the same. In theory, they are not.”
Re: Exponential Smoothing: faster and more accurate than NeuralProphet
#18Earlier quoted context omitted.
> ML (particularly DL) tends to outperform "classical" statistical time series forecasting when the data is (strongly) nonlinear, highly dimensional and large. This claim about forecasting with DL comes up a lot, but I’ve seen little evidence to back it up. Personally, I’ve never managed to have the same success others apparently have with DL time series forecasting.
It's true simply because large ANNs have a higher capacity, which is great for large, nonlinear data but less so for small datasets or simple functions. In any case, Transformers are eating ML right now and I'm actually surprised there's no "GPT-3 for time series" yet. It's technically the same problem as language modeling (that is, multi-step prediction of numerics), however, there is only a comparably little amount…
> It's technically the same problem as language modeling
You're thinking of modeling event sequences which is not strictly speaking the same as time series modeling.
Plenty of people do use LSTMs to model event sequences, using the hidden state of the model as a vector representation of processes current location walking a graph (i.e. a Users journey through a mobile app, or navigating following links on the web.)
Time series is different because the ticks of timed events are at consistent intervals and are also part of the problem being modeled. In general time series models have often been distinct from sequence models.
The reason there's no GPT-3 for any general sequence is the lack of data. Typically the vocabulary of events is much smaller than natural languages and the corpus of sequences much smaller.
Re: Exponential Smoothing: faster and more accurate than NeuralProphet
#19A minor language error: "this model does not outperform classical statistical methods neither in accuracy nor speed." should say "either" and "or".
The "Not-Neither-Nor" sequence is typical, even with regards to American English, versus British English (the Queen's English.) In either case, both are technically-correct.
As part of a double negative?
Re: Exponential Smoothing: faster and more accurate than NeuralProphet
#20This wouldn't pass peer-review of it were a paper. Major issues: - No fair hyperparametrization for Neural Prophet. They mention multiple times they used default hyperparams or ad-hoc example hyperparams. - 3/4 benchmark datasets (one they didn't finish training) where ETS outperforms is not strong evidence of all-round robustness. Benchmarks like SuperGlue for NLP combine 10 completely different tasks with more subt…
While the results don't prove the superiority convincingly, it does seem that ETS is a good candidate as a first go-to in practical applications. "In practice, practice and theory are the same. In theory, they are not.”
In theory practice and theory are the same. In practice they are not.