91–100 of 123 posts
Suppose you want to buy stocks? Would you look on a time based graph and buy according to that? Or you rather look at financial data, see earnings, profits? Wouldn't a graph that has financial performance on x-axis be more meaningful that one that has time?
What if you research real estate in a particular area? Wouldn't be square footage a better measure than time?
I'm curious why we seem convinced that this is a task that is possible or something worthy of investigation. I've worked on language models since 2018, even then it was obvious why language was a useful and transferable task. I do not at all feel the same way about general univariate time series that could have any underlying process.
It seems to me that predicting something based on time is rarely accurate and meaningful. Suppose you want to buy stocks? Would you look on a time based graph and buy according to that? Or you rather look at financial data, see earnings, profits? Wouldn't a graph that has financial performance on x-axis be more meaningful that one that has time? What if you research real estate in a particular area? Wouldn't be squar…
Things affecting financials happen through time.
I'm curious why we seem convinced that this is a task that is possible or something worthy of investigation. I've worked on language models since 2018, even then it was obvious why language was a useful and transferable task. I do not at all feel the same way about general univariate time series that could have any underlying process.
Efficiently Modeling Long Sequences with Structured State Spaces
https://www.youtube.com/watch?v=luCBXCErkCs
They made one of the best time series models and it later became one of the best language models too (Mamba).
I'm curious why we seem convinced that this is a task that is possible or something worthy of investigation. I've worked on language models since 2018, even then it was obvious why language was a useful and transferable task. I do not at all feel the same way about general univariate time series that could have any underlying process.
It seems to me that predicting something based on time is rarely accurate and meaningful. Suppose you want to buy stocks? Would you look on a time based graph and buy according to that? Or you rather look at financial data, see earnings, profits? Wouldn't a graph that has financial performance on x-axis be more meaningful that one that has time? What if you research real estate in a particular area? Wouldn't be squar…
> Would you look on a time based graph and buy according to that? Or you rather look at financial data, see earnings, profits? Things affecting financials happen through time.
I'm curious why we seem convinced that this is a task that is possible or something worthy of investigation. I've worked on language models since 2018, even then it was obvious why language was a useful and transferable task. I do not at all feel the same way about general univariate time series that could have any underlying process.
Time series data are inherently context sensitive, unlike natural languages which follow predictable grammar patterns. The patterns in time series data vary based on context. For example, flight data often show seasonal trends, while electric signals depend on the type of sensor used. There's also data that appear random, like stock data, though firms like Rentech manage to consistently find unlerlying alphas. Traini…
Also, the first realistic approximation of Solomonoff induction we achieve is going to be interesting because it will destroy the stock market.
I'm curious why we seem convinced that this is a task that is possible or something worthy of investigation. I've worked on language models since 2018, even then it was obvious why language was a useful and transferable task. I do not at all feel the same way about general univariate time series that could have any underlying process.
Fundamentally, the pre-trained model would need to learn a "world model" to predict well in distinct domains. This should be possible not regarding compute requirements and the exact architecture. After all, the physical world (down to the subatomic level) is governed by physical laws. Ilya Sutskever from OpenAI stated that next-token prediction might be enough to learn a world model (see [1]). That would imply that…
People on the AI-hype side of things tend to believe this, but I really fundamentally don't.
It's become a philosophical debate at this point (what does it mean to "understand" something, etc.)
Earlier quoted context omitted.
How data hungry is it, or what is the minimum volume of data needed before its worth investigating?
The more complex the data is, the more you need. If your values are always 5, then you need only one data point.