Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LSTM Neural Networks for Time Series Prediction

jakob-aungiers.com

Post reply on HN
Print view View original

21–23 of 23 posts

Re: LSTM Neural Networks for Time Series Prediction

#21

Post by highd »

claytonjy wrote: post #19
highd wrote: post #17

Earlier quoted context omitted.

Timeseries can 100% work like that. If you expect your timeseries data to be coming from a similar distribution that is what you do to train your LSTM. It's not just a magic box - you have to train it to encode useful features in the gate. Sure, you can prefer to use subsets of a single time series instead of multiple time series. The issue remains that it doesn't matter what your performance on training data is. You…

When doing train/test splits in a time series context like this, would forward chain backtesting (train on steps 1:n, predict n+x) be enough validation, or would you advocate for further sampling of the training set?

If you're comparing deltas (i.e. x_{n+x} - x_{n+x-1}) that might be sufficient - otherwise it's hard to tell if you're just capturing that x_{n+1} is close to x_{n}. The primary risk would be that you're putting strong structure on the datasets you're testing with, so you could be mislead. Ie what if you have:

  y = sin(t) if 0
Then you could imagine that with simply backtesting the model in front of where you're training you could run into issues - each train iteration might fix a constant frequency in the network and then it looks like it works great over each iteration, but you've never learned how to determine each frequency on-the-fly. If that happens with random backtesting from the dataset the backtesting would show that only 1/3 of the test set is fitting.

The gold standard is always a well-partitioned dataset. And if you're going to hold a meeting describing your results, or deploy a product, it's really important that the results stand up to these sorts of questions.

Re: LSTM Neural Networks for Time Series Prediction

#22

Post by highd »

rubyfan wrote: post #20
highd wrote: post #17

Earlier quoted context omitted.

Timeseries can 100% work like that. If you expect your timeseries data to be coming from a similar distribution that is what you do to train your LSTM. It's not just a magic box - you have to train it to encode useful features in the gate. Sure, you can prefer to use subsets of a single time series instead of multiple time series. The issue remains that it doesn't matter what your performance on training data is. You…

What I was interpreting the parent comment to mean two different subjects. For example, I can't train on weather data from Paris France and then expect it to be able to predict tomorrow's weather in Portland Oregon. Am I wrong on that?

You may be able to do that. That's sort of a matter of preference which you'd like to do. If the two datasets share more structure than it's more advantageous to share the network. There's also a bunch of hybrid approaches, i.e. pretraining on every city and then fine-tune each independently.

Re: LSTM Neural Networks for Time Series Prediction

#23

Post by makeset »

nomnombunty wrote: post #15
brobinson wrote: post #9

Earlier quoted context omitted.

Or Virtu having a single losing day in a six year period? https://www.bloomberg.com/news/articles/2015-02-20/high-freq... I wonder what the Sharpe Ratios of the trading systems these funds run are.

Virtu does high frequency trading so it makes sense that they don't have many down days. Also HFT strategies can have ridiculous sharpe ratios of like 100

True, although conventional metrics like Sharpe ratio or ROI are not very meaningful for HFT models, because they can't scale with any additional capital (you can safely assume they are scaled to the max). Their returns are extremely consistent, but also ultimately limited in magnitude. Rather than magical money-making machines who have cracked the "secret code" of financial markets, HFTs are essentially a fixed-cost utility service for reducing market inefficiency through improved price discovery.
Post reply on HN