Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Exponential Smoothing: faster and more accurate than NeuralProphet

github.com

Post reply on HN
Print view View original

31–40 of 44 posts

Re: Exponential Smoothing: faster and more accurate than NeuralProphet

#31

Post by ultrabear »

dcastm wrote: post #9
ren_engineer wrote: post #3

what's the consensus on machine learning vs more classical methods for time series forecasting? I know in 2018 a hybrid model won the M4 competition, obviously in this case classical still beats AI/ML https://en.wikipedia.org/wiki/Makridakis_Competitions

In the M5 competition[1], most winning solutions used LightGBM. So ML beat classical. Just a couple of the winning solutions used DL. [1] https://www.sciencedirect.com/science/article/pii/S016920702...

I would like to see the results of this ETS on the M5 Competition dataset, and see how fast it is compared to the ETS that was used as a benchmark. It goes without saying that accuracy is important, but reducing the total execution time is also pretty valuable.

Re: Exponential Smoothing: faster and more accurate than NeuralProphet

#33

Post by lr1970 »

variaga wrote: post #20
gillesjacobs wrote: post #17

Earlier quoted context omitted.

While the results don't prove the superiority convincingly, it does seem that ETS is a good candidate as a first go-to in practical applications. "In practice, practice and theory are the same. In theory, they are not.”

In theory practice and theory are the same. In practice they are not.

The difference between theory and practice is greater in theory than in practice.

Re: Exponential Smoothing: faster and more accurate than NeuralProphet

#34

Post by timy2shoes »

lr1970 wrote: post #33
variaga wrote: post #20

Earlier quoted context omitted.

In theory practice and theory are the same. In practice they are not.

The difference between theory and practice is greater in theory than in practice.

In theory there is no difference between theory and practice. In practice there is.

- Yogi Berra

Re: Exponential Smoothing: faster and more accurate than NeuralProphet

#35

Post by [deleted] »

gillesjacobs wrote: post #13

This wouldn't pass peer-review of it were a paper. Major issues: - No fair hyperparametrization for Neural Prophet. They mention multiple times they used default hyperparams or ad-hoc example hyperparams. - 3/4 benchmark datasets (one they didn't finish training) where ETS outperforms is not strong evidence of all-round robustness. Benchmarks like SuperGlue for NLP combine 10 completely different tasks with more subt…

[deleted]

Re: Exponential Smoothing: faster and more accurate than NeuralProphet

#36

Post by throwawaythekey »

time_to_smile wrote: post #16
beernet wrote: post #4

As usual in ML, the appropriate solution depends on the problem and context. ML (particularly DL) tends to outperform "classical" statistical time series forecasting when the data is (strongly) nonlinear, highly dimensional and large. The opposite holds as well. It is also important to note that accuracy is not the only relevant metric in practical applications. Explainability is of particular interest in time series…

A larger problem is that time series modeling is particularly resistant towards black box approaches since a lot of information is encoded in the model itself. Take even a simple moving average model on daily observations. Consider stock ticker data (where there are no weekends) and web traffic data (where there is an observation each day). The stock ticker data should be smoothed with a 5 day window and the web traf…

Do you know of any good resources that explain why smoothing should be used?

My intuition is that in your given example (stock prices) smoothing would probably be doing yourself a disservice as it would hide the optimal hour or day of the week to make purchases/sales.

Is it mostly related to the timeframe of your analysis and needing to trade off near term precision for longer term precision?

Re: Exponential Smoothing: faster and more accurate than NeuralProphet

#37

Post by RA_Fisher »

beernet wrote: post #4

As usual in ML, the appropriate solution depends on the problem and context. ML (particularly DL) tends to outperform "classical" statistical time series forecasting when the data is (strongly) nonlinear, highly dimensional and large. The opposite holds as well. It is also important to note that accuracy is not the only relevant metric in practical applications. Explainability is of particular interest in time series…

So true, causal inference is much more valuable than prediction which is a relatively cheap commodity.

Re: Exponential Smoothing: faster and more accurate than NeuralProphet

#38

Post by predict_addict »

gillesjacobs wrote: post #13

This wouldn't pass peer-review of it were a paper. Major issues: - No fair hyperparametrization for Neural Prophet. They mention multiple times they used default hyperparams or ad-hoc example hyperparams. - 3/4 benchmark datasets (one they didn't finish training) where ETS outperforms is not strong evidence of all-round robustness. Benchmarks like SuperGlue for NLP combine 10 completely different tasks with more subt…

This is ironic, because the Neural Prophet paper wasn’t reviewed either and didn’t benchmark it on anything other than Prophet.

Re: Exponential Smoothing: faster and more accurate than NeuralProphet

#39

Post by DoctorOetker »

timy2shoes wrote: post #34
lr1970 wrote: post #33

Earlier quoted context omitted.

The difference between theory and practice is greater in theory than in practice.

In theory there is no difference between theory and practice. In practice there is. - Yogi Berra

"Sometimes malpractice is just malpractice"

- Sigmund Freud

Re: Exponential Smoothing: faster and more accurate than NeuralProphet

#40

Post by tomwphillips »

beernet wrote: post #12
tomwphillips wrote: post #6

Earlier quoted context omitted.

> ML (particularly DL) tends to outperform "classical" statistical time series forecasting when the data is (strongly) nonlinear, highly dimensional and large. This claim about forecasting with DL comes up a lot, but I’ve seen little evidence to back it up. Personally, I’ve never managed to have the same success others apparently have with DL time series forecasting.

It's true simply because large ANNs have a higher capacity, which is great for large, nonlinear data but less so for small datasets or simple functions. In any case, Transformers are eating ML right now and I'm actually surprised there's no "GPT-3 for time series" yet. It's technically the same problem as language modeling (that is, multi-step prediction of numerics), however, there is only a comparably little amount…

Like I said, still not seen any evidence.
Post reply on HN