Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google's 200M-parameter time-series foundation model with 16k context

github.com

Post reply on HN
Print view View original

41–50 of 117 posts

Re: Google's 200M-parameter time-series foundation model with 16k context

#41

Post by teruakohatu »

perks_12 wrote: post #37
teruakohatu wrote: post #6

Earlier quoted context omitted.

What is not generally understood is that these models don’t predict egg prices or inflation in Italy. They decompose a time series into trends, seasonality and residuals. That’s what they are actually modelling. They cannot predict wars in the Middle East influencing inflation unless there is a seasonal pattern(s).

I am not familiar with time series models, but judging from your answer, it would be necessary to feed long time series into this model for it to detect trends. What is a token here? Can it, for the lack of a better example, take in all intraday movements of a stock for a day, a week, a month, etc?

I tend to avoid time series forecasting when I can help it because I find it hard to communicate to stakeholders that a neural network (or another method) is not an oracle.

If you are talking about granularity of observations, it would depend on what you are trying to predict (the price in an hour or the price in 12 months?) and how quickly you need the prediction (100ms? Tomorrow morning?). If I had infinite data I would use granularity as a hyper parameter and tune that to a level that produced the best test results.

I am for example currently using weekly averages for non-price data forecasting. I could use daily data but weekly is absolutely adequate for this purpose.

Re: Google's 200M-parameter time-series foundation model with 16k context

#42

Post by ReptileMan »

teruakohatu wrote: post #6
EmilStenstrom wrote: post #4

I somehow find the concept of a general time series model strange. How can the same model predict egg prices in Italy, and global inflation in a reliable way? And how would you even use this model, given that there are no explanations that help you trust where the prediction comes from…

What is not generally understood is that these models don’t predict egg prices or inflation in Italy. They decompose a time series into trends, seasonality and residuals. That’s what they are actually modelling. They cannot predict wars in the Middle East influencing inflation unless there is a seasonal pattern(s).

It is the Middle East. Wars are always in season. And supply is more than the demand.

Re: Google's 200M-parameter time-series foundation model with 16k context

#43

Post by d--b »

nico wrote: post #22
d--b wrote: post #15

Earlier quoted context omitted.

The main issue is that people do use them to predict bitcoin prices intraday and that sort of things.

Is it an issue because it works, or because it doesn’t? Or because it’s bitcoin? I genuinely want to know. Thank you

It is an issue because bitcoin is highly unpredictable.

These tools are good at predicting timeseries that are in fact quite predictable. Like insurances will use this to estimate the number of people who will die from cancer in the next year, the year after that, and so on up to 50 years in the future. The model will extrapolate the progresses made in cancer treatment from the current trend, etc. It is a prediction, cause it's still possible that a breakthrough comes in and suddenly people don't die from a certain form of cancer, but generally it should be roughly correct.

Bitcoin prices are a lot more chaotic, influenced by a ton of unrelated events that shape its path a certain way. There is absolutely no certainty that studying the shape of its past evolution will help in any way understand its future evolution.

Of course here I mean by studying its price alone. If you add more information, like who's behind each trend and why, you have a much better sense of what could happen next.

Re: Google's 200M-parameter time-series foundation model with 16k context

#46

Post by annie511266728 »

EmilStenstrom wrote: post #4

I somehow find the concept of a general time series model strange. How can the same model predict egg prices in Italy, and global inflation in a reliable way? And how would you even use this model, given that there are no explanations that help you trust where the prediction comes from…

It’s not really predicting “egg prices” or “inflation” — it’s mostly fitting patterns that happen to show up in those series.

The problem isn’t domain generalization, it’s that we keep pretending these models have any notion of what the data means.

People ask how one model can understand everything, but that assumes there’s any understanding involved at all.

At some point you have to ask: how much of “forecasting” is actually anything more than curve fitting with better marketing?

Re: Google's 200M-parameter time-series foundation model with 16k context

#47

Post by eru »

cenamus wrote: post #27
eru wrote: post #24

Earlier quoted context omitted.

> How can the same model predict egg prices in Italy, and global inflation in a reliable way? How can the same lossy compression algorithm (eg JPG) compress pictures of everything in a reliable way?

It can't compress pictures of everything in a reliable way. Text and anything with lots of high frequency components looks terrible

It still doesn't pretty well on text. And we have newer formats and ideas that would also deal with that. (To be really dead simple: have a minimal container format that decides between png or jpg, use png for text.)

However: white noise is where it really struggles. But real pictures of the real world don't look like white noise. Even though in some sense white noise is the most common type of picture a priori.

Similar for real world time series: reality mostly doesn't look like white noise.

Re: Google's 200M-parameter time-series foundation model with 16k context

#48

Post by lordgrenville »

teruakohatu wrote: post #6
EmilStenstrom wrote: post #4

I somehow find the concept of a general time series model strange. How can the same model predict egg prices in Italy, and global inflation in a reliable way? And how would you even use this model, given that there are no explanations that help you trust where the prediction comes from…

What is not generally understood is that these models don’t predict egg prices or inflation in Italy. They decompose a time series into trends, seasonality and residuals. That’s what they are actually modelling. They cannot predict wars in the Middle East influencing inflation unless there is a seasonal pattern(s).

That's what traditional time-series modelling does. This is a foundational model, which means it's just a neural network trained on lots of time series. (So maybe OP's question still stands? But it's the same question as "how can LLMs be good at so many different kinds of conversations?")

Re: Google's 200M-parameter time-series foundation model with 16k context

#50

Post by graemep »

teruakohatu wrote: post #6
EmilStenstrom wrote: post #4

I somehow find the concept of a general time series model strange. How can the same model predict egg prices in Italy, and global inflation in a reliable way? And how would you even use this model, given that there are no explanations that help you trust where the prediction comes from…

What is not generally understood is that these models don’t predict egg prices or inflation in Italy. They decompose a time series into trends, seasonality and residuals. That’s what they are actually modelling. They cannot predict wars in the Middle East influencing inflation unless there is a seasonal pattern(s).

Do these models predict on just a single time series then?

it is far more useful for predictions to look for correlations between time series. This is far more complex than looking for correlations in general because most time series trend up or down and therefore correlate.

Post reply on HN