Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google's 200M-parameter time-series foundation model with 16k context

github.com

Post reply on HN
Print view View original

31–40 of 117 posts

Re: Google's 200M-parameter time-series foundation model with 16k context

#31

Post by tgv »

mikkom wrote: post #23
jdthedisciple wrote: post #20

Let me be blunt: Shannon would tell us that time forecasting is bullshit: There is infinitely more entropy in the real world out there than any model can even remotely capture. The world is not minecraft.

Yeah all weather forecasts are just magic

Weather forecasts are notoriously iffy, and accuracy drops with time, but we understand the physics behind it (to a large extent). There's also a lot of fine-grained data available. For some arbitrary time series, there's only one data sequence, and the model is unknown. Extrapolation then becomes a lot more magical.

Re: Google's 200M-parameter time-series foundation model with 16k context

#32

Post by OliverGuy »

EmilStenstrom wrote: post #3

Here is the link to the blogpost, that actually describe what this is: https://github.com/google-research/timesfm?tab=readme-ov-fil...

Wish they gave some numbers for total GPU hours to train this model, seems comparatively tiny when compared to LLMs so interested to know how close this is to something trainable by your average hobbyist/university/small lab

Re: Google's 200M-parameter time-series foundation model with 16k context

#34

Post by jofzar »

cybrox wrote: post #16
teruakohatu wrote: post #6

Earlier quoted context omitted.

What is not generally understood is that these models don’t predict egg prices or inflation in Italy. They decompose a time series into trends, seasonality and residuals. That’s what they are actually modelling. They cannot predict wars in the Middle East influencing inflation unless there is a seasonal pattern(s).

Wars in the middle east seem to have increasingly regular patterns tied to stock market opening hours, unfortunately.

I mean it's super obvious, it's directly tied to scrubs popularity.

New season of scrubs = new war in the middle east.

Re: Google's 200M-parameter time-series foundation model with 16k context

#36

Post by Tarq0n »

dash2 wrote: post #21

So the time series are provided with no context? It's just trained on lots of sets of numbers? Then you give it a new set of numbers and it guesses the rest, again with no context? My guess as to how this would work: the machine will first guess from the data alone if this is one of the categories it has already seen/inferred (share prices, google trend cat searches etc.) Then it'll output a plausible completion for…

If it works for predicting the next token in a very long stream of tokens, why not. The question is what architecture and training regimen it needs to generalize.

Re: Google's 200M-parameter time-series foundation model with 16k context

#37

Post by perks_12 »

teruakohatu wrote: post #6
EmilStenstrom wrote: post #4

I somehow find the concept of a general time series model strange. How can the same model predict egg prices in Italy, and global inflation in a reliable way? And how would you even use this model, given that there are no explanations that help you trust where the prediction comes from…

What is not generally understood is that these models don’t predict egg prices or inflation in Italy. They decompose a time series into trends, seasonality and residuals. That’s what they are actually modelling. They cannot predict wars in the Middle East influencing inflation unless there is a seasonal pattern(s).

I am not familiar with time series models, but judging from your answer, it would be necessary to feed long time series into this model for it to detect trends. What is a token here? Can it, for the lack of a better example, take in all intraday movements of a stock for a day, a week, a month, etc?

Re: Google's 200M-parameter time-series foundation model with 16k context

#38

Post by magimas »

raghavMultilipi wrote: post #30

This has been around a few months now, has anyone built anything on it?

we did some internal tests. The quality isn't bad, it works quite well. But it's essentially on the same level of an ARIMA model trained on the data just much bigger and slower.

So in my opinion it currently falls into a kind of void. If your use case is worth predicting and you put a data scientist on it, you're better off just training cheaper ARIMA models.

Re: Google's 200M-parameter time-series foundation model with 16k context

#39

Post by OliverGuy »

OliverGuy wrote: post #32
EmilStenstrom wrote: post #3

Here is the link to the blogpost, that actually describe what this is: https://github.com/google-research/timesfm?tab=readme-ov-fil...

Wish they gave some numbers for total GPU hours to train this model, seems comparatively tiny when compared to LLMs so interested to know how close this is to something trainable by your average hobbyist/university/small lab

Edit, it looks like the paper does

TPUv5e with 16 tensor cores for 2 days for the 200M param model.

Claude reckons this is 60 hours on a 8xA100 rig, so very accessibile compared to LLMs for smaller labs

Post reply on HN