Somehow I missed that one. Are there any competition on this? I always had difficulties with ML and time series, I'll need to try that out.
there is TabPFN [1] which also has time series capabilities.
Google's 200M-parameter time-series foundation model with 16k context
51–60 of 117 posts
Re: Google's 200M-parameter time-series foundation model with 16k context
#52This has been around a few months now, has anyone built anything on it?
we did some internal tests. The quality isn't bad, it works quite well. But it's essentially on the same level of an ARIMA model trained on the data just much bigger and slower. So in my opinion it currently falls into a kind of void. If your use case is worth predicting and you put a data scientist on it, you're better off just training cheaper ARIMA models.
That is disappointing. One would say that with all the budget and compute, Google would be able to create something that beats methods from 70s. Maybe we are hitting some hard limits.
Maybe it would be better to train an LLM with various tuning methodologies and make a dedicated ARIMA agent. You throw in data, some metadata and requested window of forecast. Out comes parameters for "optimal" conventional model.
Re: Google's 200M-parameter time-series foundation model with 16k context
#53Earlier quoted context omitted.
Wars in the middle east seem to have increasingly regular patterns tied to stock market opening hours, unfortunately.
I mean it's super obvious, it's directly tied to scrubs popularity. New season of scrubs = new war in the middle east.
Wow, I didn't know. Thank you! Such a great show.
Re: Google's 200M-parameter time-series foundation model with 16k context
#54Earlier quoted context omitted.
It can't compress pictures of everything in a reliable way. Text and anything with lots of high frequency components looks terrible
It still doesn't pretty well on text. And we have newer formats and ideas that would also deal with that. (To be really dead simple: have a minimal container format that decides between png or jpg, use png for text.) However: white noise is where it really struggles. But real pictures of the real world don't look like white noise. Even though in some sense white noise is the most common type of picture a priori. Simi…
White noise is random, so it's incompressible by definition. By JPG or by any other method no matter how clever.
Re: Google's 200M-parameter time-series foundation model with 16k context
#55Earlier quoted context omitted.
It still doesn't pretty well on text. And we have newer formats and ideas that would also deal with that. (To be really dead simple: have a minimal container format that decides between png or jpg, use png for text.) However: white noise is where it really struggles. But real pictures of the real world don't look like white noise. Even though in some sense white noise is the most common type of picture a priori. Simi…
White noise is random, so it's incompressible by definition. By JPG or by any other method no matter how clever.
[deleted]
Re: Google's 200M-parameter time-series foundation model with 16k context
#56Let me be blunt: Shannon would tell us that time forecasting is bullshit: There is infinitely more entropy in the real world out there than any model can even remotely capture. The world is not minecraft.
> Shannon would tell us that time forecasting is bullshit
If you're trying to forecast random data, then yes, it's bullshit. Otherwise you have a chance.
Re: Google's 200M-parameter time-series foundation model with 16k context
#57I somehow find the concept of a general time series model strange. How can the same model predict egg prices in Italy, and global inflation in a reliable way? And how would you even use this model, given that there are no explanations that help you trust where the prediction comes from…
What is not generally understood is that these models don’t predict egg prices or inflation in Italy. They decompose a time series into trends, seasonality and residuals. That’s what they are actually modelling. They cannot predict wars in the Middle East influencing inflation unless there is a seasonal pattern(s).
What makes these models different from models used for e.g. audio?
Or other low-dimensional time domain signals?
Re: Google's 200M-parameter time-series foundation model with 16k context
#58Earlier quoted context omitted.
It still doesn't pretty well on text. And we have newer formats and ideas that would also deal with that. (To be really dead simple: have a minimal container format that decides between png or jpg, use png for text.) However: white noise is where it really struggles. But real pictures of the real world don't look like white noise. Even though in some sense white noise is the most common type of picture a priori. Simi…
White noise is random, so it's incompressible by definition. By JPG or by any other method no matter how clever.
I have a very peculiar coin. With 1% probability it turns up heads and with 99% probability it turns up tails.
A string of flips is random, but it's very compressible.
In any case, my point was that reality ain't uniformly random. And not only that: pretty much anything you can point your camera at shares enough similarity in their distribution that we pretty much have universal compression algorithms for real world data.
Re: Google's 200M-parameter time-series foundation model with 16k context
#59Earlier quoted context omitted.
we did some internal tests. The quality isn't bad, it works quite well. But it's essentially on the same level of an ARIMA model trained on the data just much bigger and slower. So in my opinion it currently falls into a kind of void. If your use case is worth predicting and you put a data scientist on it, you're better off just training cheaper ARIMA models.
That is disappointing. One would say that with all the budget and compute, Google would be able to create something that beats methods from 70s. Maybe we are hitting some hard limits. Maybe it would be better to train an LLM with various tuning methodologies and make a dedicated ARIMA agent. You throw in data, some metadata and requested window of forecast. Out comes parameters for "optimal" conventional model.
I think this could be an interesting read for you, I read it last week and it kind of argues the same points:
https://shakoist.substack.com/p/against-time-series-foundati...
Re: Google's 200M-parameter time-series foundation model with 16k context
#60I somehow find the concept of a general time series model strange. How can the same model predict egg prices in Italy, and global inflation in a reliable way? And how would you even use this model, given that there are no explanations that help you trust where the prediction comes from…
It’s not really predicting “egg prices” or “inflation” — it’s mostly fitting patterns that happen to show up in those series. The problem isn’t domain generalization, it’s that we keep pretending these models have any notion of what the data means. People ask how one model can understand everything, but that assumes there’s any understanding involved at all. At some point you have to ask: how much of “forecasting” is…
"curve-fitting" has a long history (centuries old) and could be regarded more as a numerical method issue.
Rigorous understanding of what is over fitting, techniques to avoid it and select the right complexity of the model, etc, are much newer. This is a statistical issue.
My point is that forecasting isn't curve fitting, even thought curve fitting is one element of it.