Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Chronos: Learning the Language of Time Series

arxiv.org

Post reply on HN
Print view View original

31–40 of 63 posts

Re: Chronos: Learning the Language of Time Series

#31

Post by dist-epoch »

lamename wrote: post #29
dist-epoch wrote: post #25

Earlier quoted context omitted.

Tokenisation turns a continuous signal into a normalized discrete vocabulary: stock "went up a lot", "went up a little", "stayed flat". This smooths out noise and simplifies matching up similar but not identical signals. > We tokenize text because text isn't numbers. Text is actually numbers. People tried inputting UTF8 directly into transformers, but it doesn't work that well. Karpathy explains why: https://www.yout…

Interesting. Can you explain how this is superior and/or different from traditional DSP filters or other non-tokenization tricks in the signal processing field?

Traditional DSP filters still output a continuous signal. And it's a well-explored domain, hard to imagine any low-hanging fruit there.

My intuition is the following: transformers work really well for text, so we could try turning a time series into a "story" (limited vocabulary) and see what happens.

Re: Chronos: Learning the Language of Time Series

#32

Post by melondonkey »

As a practitioner the most impactful library for time series has been brms, which basically gives you syntactic sugar for creating statistical models in Stan. Checks all the boxes including probabilistic forecasts, multiple link functions for the likelihood including weiner, gamma, Gaussian, student t, binomial, zero-inflated and hurdle models. Also has auto-regressive and ordinal predictors and you actually learn something from your data.

I find a lot of these ML and DL libraries to be harder to troubleshoot beyond blind hyperparameter tuning whereas with stats I can tweak model, modify likelihood, etc. There’s also a lot of high value problems that have few data points these libraries tend to want at least daily data.

Re: Chronos: Learning the Language of Time Series

#33

Post by prlin »

dist-epoch wrote: post #25
meow_cat wrote: post #20

Maybe I'm missing something obvious, but what is the idea behind quantizing and tokenizing time series? We tokenize text because text isn't numbers. In the case of time series, we're... turning numbers into less precise numbers? The benefit of scaling and centering is trivial and i guess all timeseries ML does it, but I don't see why we need a token after that.

Tokenisation turns a continuous signal into a normalized discrete vocabulary: stock "went up a lot", "went up a little", "stayed flat". This smooths out noise and simplifies matching up similar but not identical signals. > We tokenize text because text isn't numbers. Text is actually numbers. People tried inputting UTF8 directly into transformers, but it doesn't work that well. Karpathy explains why: https://www.yout…

> Text is actually numbers

Text can be represented by numbers but they aren't the same datatype. They don't support the same operations (addition, subtraction, multiplication, etc).

Re: Chronos: Learning the Language of Time Series

#34

Post by mikkom »

hackerlight wrote: post #24
wenc wrote: post #11

Earlier quoted context omitted.

They are comparing a non-ensembled transformer model with an ensemble of simple linear models. It's not surprising that the ensemble models of linear time series models will do well, since ensembles optimize for the bias-variance trade-off. Transformer/ML models by themselves have a tendency to overfit past patterns. They pick up more signal in the patterns, but they also pick up spurious patterns. They're low bias b…

> ensemble of transformer models Isn't that just dropout?

No. Why do you think so?

Re: Chronos: Learning the Language of Time Series

#35

Post by kfor »

sampo wrote: post #21

Amazon's older time series forecasting system DeepAR, has supported using external regressors since 2018 [1]. From this new Chronos paper, I didn't find any mention of external regressors. [1] https://aws.amazon.com/blogs/machine-learning/amazon-sagemak...

They do mention covariates in section 6.1 - specifically how this method doesn’t support them but ideas on how they could in the future such as via stacking:

> In this work, we have focused on univariate time series forecasting since it constitutes the most common of real-world time series use-cases. Nevertheless, practical forecasting tasks often involve additional information that must be taken into account. One example involves covariates, that can be either time-independent (e.g., color of the product) or time-varying (e.g., on which days the product is on sale). Another closely related problem is multivariate forecasting, where historic values of one time series (e.g., interest rates) can influence the forecast for another time series (e.g., housing prices). The number of covariates or multivariate dimensions can vary greatly across tasks, which makes it challenging to train a single model that can handle all possible combinations. A possible solution may involve training task-specific adaptors that inject the covariates into the pretrained forecasting model (Rahman et al., 2020). As another option, we can build stacking ensembles (Ting & Witten, 1997) of Chronos and other light-weight models that excel at handling covariates such as LightGBM (Ke et al., 2017).

Re: Chronos: Learning the Language of Time Series

#37

Post by sampo »

kfor wrote: post #35
sampo wrote: post #21

Amazon's older time series forecasting system DeepAR, has supported using external regressors since 2018 [1]. From this new Chronos paper, I didn't find any mention of external regressors. [1] https://aws.amazon.com/blogs/machine-learning/amazon-sagemak...

They do mention covariates in section 6.1 - specifically how this method doesn’t support them but ideas on how they could in the future such as via stacking: > In this work, we have focused on univariate time series forecasting since it constitutes the most common of real-world time series use-cases. Nevertheless, practical forecasting tasks often involve additional information that must be taken into account. One ex…

Ah. Thank you. The same concept goes under different names, so one needs to search for all of "exogenous variables", "external regressors", "external factors" and "covariates".

Re: Chronos: Learning the Language of Time Series

#39

Post by lamename »

dist-epoch wrote: post #31
lamename wrote: post #29

Earlier quoted context omitted.

Interesting. Can you explain how this is superior and/or different from traditional DSP filters or other non-tokenization tricks in the signal processing field?

Traditional DSP filters still output a continuous signal. And it's a well-explored domain, hard to imagine any low-hanging fruit there. My intuition is the following: transformers work really well for text, so we could try turning a time series into a "story" (limited vocabulary) and see what happens.

Like this or something different?

https://github.com/gzerveas/mvts_transformer

Post reply on HN