Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google's 200M-parameter time-series foundation model with 16k context

github.com

Post reply on HN
Print view View original

81–90 of 117 posts

Re: Google's 200M-parameter time-series foundation model with 16k context

#81

Post by amelius »

carschno wrote: post #80
amelius wrote: post #57

Earlier quoted context omitted.

What makes these models different from models used for e.g. audio? Or other low-dimensional time domain signals?

You could abstract speech or other audio as a series of sounds, where time is indeed a factor. Speech, however, has patterns that are more similar to written language than to seasonal patterns that are typically assumed in time series. While trained on different data, the architecture of TimesFM is actually similar to LLMs. But not identical, as pointed out at https://research.google/blog/a-decoder-only-foundation-mo…

If "seasonal patterns" is the thing that differentiates between these two data sources, then perhaps time series models should be called seasonal models?

Re: Google's 200M-parameter time-series foundation model with 16k context

#82

Post by robot-wrangler »

dist-epoch wrote: post #76
robot-wrangler wrote: post #74

Earlier quoted context omitted.

This seemed like a good answer at first. But on further thought, images on the whole really do seem to have quite a bit more standard structure / "grammar" to exploit compared to arbitrary time-series. Many images are of the world, where there is gravity so you might see preponderance of blobs at the bottom, or the repetitive types like people, animals, faces, eyes. Wildly abstract images still have some continuity,…

> It's less clear to me how mixing sensor data / financial data / anything else together could be helpful. Because many of these have the same underlying causal structures - humans doing things, weather correlations, holidays. Well studied behavioral stuff like "the stock market takes the stairs up and the elevator down" which is not really captured by "traditional" modelling tools. I'm sure people will be doing mech…

> Because many of these have the same underlying causal structures - humans doing things, weather correlations, holidays.

Or, you know, maybe they aren't. Thermometers and photon counts are related to weather sometimes, but not holidays. Holidays are related to traffic sensors and to markets, but not Geiger counters.

> Well studied behavioral stuff like "the stock market takes the stairs up and the elevator down" which is not really captured by "traditional" modelling tools.

Prices are the opposite, up like a shot during shocks, falling slowly like a feather. So that particular pattern seems like a great example of over-fitting danger and why you wouldn't expect mixing series of different types to be work very well.

Re: Google's 200M-parameter time-series foundation model with 16k context

#84

Post by bitshiftfaced »

Foobar8568 wrote: post #2

Somehow I missed that one. Are there any competition on this? I always had difficulties with ML and time series, I'll need to try that out.

There are some other transformer based models on the GIFT leaderboard: https://huggingface.co/spaces/Salesforce/GIFT-Eval

Re: Google's 200M-parameter time-series foundation model with 16k context

#85

Post by esafak »

torginus wrote: post #78
dist-epoch wrote: post #76

Earlier quoted context omitted.

> It's less clear to me how mixing sensor data / financial data / anything else together could be helpful. Because many of these have the same underlying causal structures - humans doing things, weather correlations, holidays. Well studied behavioral stuff like "the stock market takes the stairs up and the elevator down" which is not really captured by "traditional" modelling tools. I'm sure people will be doing mech…

Personally, coming from an EE background and not finance or statistics, I would go about identifying these patterns with an Signals & Systems toolbox, like systems identification, various matched filters/classifiers. This might be a totall wrong approach, but I think it might make sense to try to model a matched filter based on previous stock selloff/bullrun trigger events, and then see if the it has any predictive a…

You're crafting features. The modern approach to ML (deep learning) is to use over-parameterized models and let them learn the features. Perhaps you remember this? https://www.nytimes.com/2012/06/26/technology/in-a-big-netwo...

Re: Google's 200M-parameter time-series foundation model with 16k context

#86

Post by [deleted] »

EmilStenstrom wrote: post #4

I somehow find the concept of a general time series model strange. How can the same model predict egg prices in Italy, and global inflation in a reliable way? And how would you even use this model, given that there are no explanations that help you trust where the prediction comes from…

[deleted]

Re: Google's 200M-parameter time-series foundation model with 16k context

#87

Post by Forgeties79 »

guntars wrote: post #79
jcelerier wrote: post #63

Earlier quoted context omitted.

> They cannot predict wars in the Middle East influencing inflation unless there is a seasonal pattern(s). well...

The Middle East war season is upon us once again

Born too soon to deploy to the Middle East.

Born too late to deploy to the Middle East.

Born just in time to deploy to the Middle East.

Re: Google's 200M-parameter time-series foundation model with 16k context

#89

Post by dist-epoch »

robot-wrangler wrote: post #82
dist-epoch wrote: post #76

Earlier quoted context omitted.

> It's less clear to me how mixing sensor data / financial data / anything else together could be helpful. Because many of these have the same underlying causal structures - humans doing things, weather correlations, holidays. Well studied behavioral stuff like "the stock market takes the stairs up and the elevator down" which is not really captured by "traditional" modelling tools. I'm sure people will be doing mech…

> Because many of these have the same underlying causal structures - humans doing things, weather correlations, holidays. Or, you know, maybe they aren't. Thermometers and photon counts are related to weather sometimes, but not holidays. Holidays are related to traffic sensors and to markets, but not Geiger counters. > Well studied behavioral stuff like "the stock market takes the stairs up and the elevator down" whi…

Electricity demand is influenced very strongly by holidays, strongly by weather and from weak to strong by geopolitics (depending on location).

The model will have a library of patterns, and will be able to pattern match subtle ones to deduce "this time series has the kind of micro-patterns which appear in strongly weather influenced time-series", and use this to activate the weather pattern cluster.

To use your example, when served thermometer data, the model notices that the holiday pattern cluster doesn't activate/match at all, and will ignore it.

And then it makes sense to train it on the widest possible time series, so it can build a vast library of patterns and find correlations of activation between them.

Re: Google's 200M-parameter time-series foundation model with 16k context

#90

Post by morkalork »

jcelerier wrote: post #63
teruakohatu wrote: post #6

Earlier quoted context omitted.

What is not generally understood is that these models don’t predict egg prices or inflation in Italy. They decompose a time series into trends, seasonality and residuals. That’s what they are actually modelling. They cannot predict wars in the Middle East influencing inflation unless there is a seasonal pattern(s).

> They cannot predict wars in the Middle East influencing inflation unless there is a seasonal pattern(s). well...

Next you'll suggest something looney like a correlation with the 11-year solar cycle!

(for those who are lost: https://x.com/onionweigher/status/1936630237208469898)

Post reply on HN