Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Time Series Prediction Using LSTM Deep Neural Networks

altumintelligence.com

Post reply on HN
Print view View original

41–50 of 72 posts

Re: Time Series Prediction Using LSTM Deep Neural Networks

#41

Post by beagle3 »

qeternity wrote: post #35
beagle3 wrote: post #17

Earlier quoted context omitted.

.. because you buy at the price, and sell at the price (spread and fees ignored for now). Which means, regardless of your philosophy, you are predicting a price change - a long signal is a prediction for positive price change; a short signal is a prediction for a negative price change. If that wasn’t true, your system would not be able to profit. Predicting price change and predicting price are semantically equivalen…

Given less than 100% certainty, traders don't want to predict price, they want to predict future distribution of price over some time period. Source: hedge fund trader

True. That’s still considered a prediction of price among my trading colleagues.

Re: Time Series Prediction Using LSTM Deep Neural Networks

#42

Post by beagle3 »

jarym wrote: post #21
beagle3 wrote: post #17

Earlier quoted context omitted.

.. because you buy at the price, and sell at the price (spread and fees ignored for now). Which means, regardless of your philosophy, you are predicting a price change - a long signal is a prediction for positive price change; a short signal is a prediction for a negative price change. If that wasn’t true, your system would not be able to profit. Predicting price change and predicting price are semantically equivalen…

Predicting price means you’re predicting one variable with no idea of hot likely you are to be wrong and how wrong you’re likely to be and says nothing of where your expectations are for price to go after. It is semantically different to say: if price goes to Y then you have odds that it will then go to Target 1 and then slightly lower odds it goes to Target 2.

Do you long now, or short now, or neither? If you go long, you’ve predicted price goes higher. Regardless of what you think about the entire path. Mathematically nothing else makes sense.

Re: Time Series Prediction Using LSTM Deep Neural Networks

#43

Post by laichzeit0 »

zwaps wrote: post #28

I find it interesting that Computer Scientists are basically rediscovering statistics. Now when predicting time series, an issue is that most model (like ARIMA, GARCH etc.) are short-memory processes. When you look at the full-series prediction of LSTMs, you observe the same thing. So in terms of Time Series, Machine Learning is currently in the mid to late 80's compared to Financial Econometrics. So if you are a CS,…

I don't think it's entirely fair to say "Computer Scientists are basically rediscovering statistics". LSTMs are used beyond just time series prediction. It is also quite common in language modelling tasks, which is also a sequence modelling task, and where it works quite well. I'm not familiar at all with using GARCH/ARIMA for something like this.

Also, with neural networks it's very easy and natural to build complex models where different "layers" perform different tasks. So an LSTM can very easily be extended to work bi-directionally (taking data from the beginning of the sequence, and the end of the sequence), adding things like attention, using word-vectors before the recurrent network or just using a character model.

What are the statistical equivalents for this? Because most of the papers on this topic seem to come from Computer Science. Take a look at the epilogue of [1] for a thorough discussion on where statistical theory needs to catch up.

[1] Computer Age Statistical Inference - Efron, Hastie.

Re: Time Series Prediction Using LSTM Deep Neural Networks

#44

Post by RA_Fisher »

Eridrus wrote: post #39
RA_Fisher wrote: post #32

Earlier quoted context omitted.

It's been amazing to watch CS (really the Python community, save statsmodels and patsy) discover statistics. For a while I thought perhaps it was me and statistics that was "behind." Over time I realized that it was mostly re-invention of old ideas: one-hot encoding = dummy variables, neural networks approximating polynomial regression, etc. I decided to double-down on statistics and it's really paid off. NN / random…

I'm curious what resources you found useful to learn stats modelling and what sorts of approaches have been useful. On one hand, it's almost a tautoloy that specific models should be better than general models, but I worked on some 2d time series classification with a statistician and afterwards, for kicks, I replaced the entire thing with a CNN+LSTM and it worked just as well as the whole complicated model he had co…

I highly recommend this econometrics text for getting started with statistics: https://www.amazon.com/Principles-Econometrics-5th-Carter-Hi...

For modeling I found Wooldridge's panel and cross-section data book very useful: https://www.amazon.com/Econometric-Analysis-Cross-Section-Pa...

Greene is a really useful reference text: https://www.amazon.com/Econometric-Analysis-8th-William-Gree...

For advanced stats theory, I recommend Casella and Berger https://www.amazon.com/Statistical-Inference-George-Casella/...

Hope that helps!

The more specific a model can be made to the problem at hand, the better it'll perform. Supervised ML models are great starting / baseline models.

Re: Time Series Prediction Using LSTM Deep Neural Networks

#45

Post by jarym »

beagle3 wrote: post #40
beagle3 wrote: post #17

Earlier quoted context omitted.

.. because you buy at the price, and sell at the price (spread and fees ignored for now). Which means, regardless of your philosophy, you are predicting a price change - a long signal is a prediction for positive price change; a short signal is a prediction for a negative price change. If that wasn’t true, your system would not be able to profit. Predicting price change and predicting price are semantically equivalen…

Replying to myself instead of all three replies so far: People, prediction is a general term. Many predictors come with accuracy estimates (and outside of finance, often prediction bounds). But even if it was only one number - if you have good prediction of the expected price change, that could be sufficient to trade as it encompasses, by definition, the sun of probability of different outcomes times their magnitude.…

Right, but that's not what this article (and several others I've seen) are doing - they are measuring the performance of some AI by putting on some unnecessary (and quite likely impossible) constraints on what it is supposed to be outputting.

All I'm pointing out is that measuring any stock trading algo by treating it as a regression problem for the exact next time step is a naive approach - that's not the same thing as what human traders are doing.

Anyway, if you are trading then I wish you lots of success.

Re: Time Series Prediction Using LSTM Deep Neural Networks

#46

Post by curiousgal »

RA_Fisher wrote: post #32
zwaps wrote: post #28

I find it interesting that Computer Scientists are basically rediscovering statistics. Now when predicting time series, an issue is that most model (like ARIMA, GARCH etc.) are short-memory processes. When you look at the full-series prediction of LSTMs, you observe the same thing. So in terms of Time Series, Machine Learning is currently in the mid to late 80's compared to Financial Econometrics. So if you are a CS,…

It's been amazing to watch CS (really the Python community, save statsmodels and patsy) discover statistics. For a while I thought perhaps it was me and statistics that was "behind." Over time I realized that it was mostly re-invention of old ideas: one-hot encoding = dummy variables, neural networks approximating polynomial regression, etc. I decided to double-down on statistics and it's really paid off. NN / random…

Because saying "relevant username" is frowned upon I'll just point out that R A Fisher is "a genius who almost single-handedly created the foundations for modern statistical science"[0]

0.https://en.m.wikipedia.org/wiki/Ronald_Fisher

Re: Time Series Prediction Using LSTM Deep Neural Networks

#47

Post by cs702 »

RA_Fisher wrote: post #32
zwaps wrote: post #28

I find it interesting that Computer Scientists are basically rediscovering statistics. Now when predicting time series, an issue is that most model (like ARIMA, GARCH etc.) are short-memory processes. When you look at the full-series prediction of LSTMs, you observe the same thing. So in terms of Time Series, Machine Learning is currently in the mid to late 80's compared to Financial Econometrics. So if you are a CS,…

It's been amazing to watch CS (really the Python community, save statsmodels and patsy) discover statistics. For a while I thought perhaps it was me and statistics that was "behind." Over time I realized that it was mostly re-invention of old ideas: one-hot encoding = dummy variables, neural networks approximating polynomial regression, etc. I decided to double-down on statistics and it's really paid off. NN / random…

True.

On the other hand, the "more ignorant CS approach" has produced impressive achievements in language tasks (e.g., translation), visual tasks (e.g., image generation), game playing tasks (e.g., Go), agent-in-virtual-world tasks (e.g., DOTA), and robot-in-real-world tasks (e.g., self-driving cars).

Academic statistics departments often seem to be "20 years behind" on all those fronts...

Re: Time Series Prediction Using LSTM Deep Neural Networks

#48

Post by md2be »

zwaps wrote: post #28

I find it interesting that Computer Scientists are basically rediscovering statistics. Now when predicting time series, an issue is that most model (like ARIMA, GARCH etc.) are short-memory processes. When you look at the full-series prediction of LSTMs, you observe the same thing. So in terms of Time Series, Machine Learning is currently in the mid to late 80's compared to Financial Econometrics. So if you are a CS,…

I agree with your sentiments, but there is a contribution that the CS departments made that the statistics, math, Econ (as in econometrics departments) seemed to have overlooked. I remember going to each of these departments in 2002 and asking them why don’t we split the data sets to train and update the coefficients and automate the process. The answer was always the same “that’s trivial and adds nothing to the field”.

Re: Time Series Prediction Using LSTM Deep Neural Networks

#49

Post by md2be »

cs702 wrote: post #47
RA_Fisher wrote: post #32

Earlier quoted context omitted.

It's been amazing to watch CS (really the Python community, save statsmodels and patsy) discover statistics. For a while I thought perhaps it was me and statistics that was "behind." Over time I realized that it was mostly re-invention of old ideas: one-hot encoding = dummy variables, neural networks approximating polynomial regression, etc. I decided to double-down on statistics and it's really paid off. NN / random…

True. On the other hand, the "more ignorant CS approach" has produced impressive achievements in language tasks (e.g., translation), visual tasks (e.g., image generation), game playing tasks (e.g., Go), agent-in-virtual-world tasks (e.g., DOTA), and robot-in-real-world tasks (e.g., self-driving cars). Academic statistics departments often seem to be "20 years behind" on all those fronts...

No doubt that club of statistical significance held back many statisticians.

Re: Time Series Prediction Using LSTM Deep Neural Networks

#50

Post by md2be »

djhworld wrote: post #24

I'm currently learning machine learning at the most basic level, this is the sort of stuff I want to work towards though I deal with time series data a lot at work, I work in broadcasting/media and 99% of the time the data is fairly "predictable" and follows a regular daily pattern, peppered with the odd spikes during big, unpredicatble news events.

Time series analysis requires the data to be stationary.
Post reply on HN