Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Using Reinforcement Learning in the Algorithmic Trading Problem

arxiv.org

Post reply on HN
Print view View original

101–110 of 139 posts

Re: Using Reinforcement Learning in the Algorithmic Trading Problem

#101

Post by traK6Dcm »

p1esk wrote: post #90
lumost wrote: post #80

Earlier quoted context omitted.

There are some ML problems where it is fundamentally impossible to use historical data to make accurate forward looking predictions as its not IID. These fields require you very carefully capture data on sub-optimal choices. In the case of trading this means making explicitly bad trading decisions some portion of the time, and teams that have done this at any scale are unlikely to share the data. In the case of tradi…

I don't get it - if you have accurate historical data, how is this different from having access to current real-time data? Why can't you pretend you live 20 years in the past and use the data you have as if it were real-time?

Data distribution shift. The market changes over time and your current data does not come from the same distribution as old data. That limits the amount of data you can use for training and testing. You need to be very careful not to overfit. That's especially true for something like daily or hourly data - there isn't much data to begin with and you won't have much left if you look at only a few weeks or months. Market data already has a low signal/noise ratio to begin with, so you need a good chunk of data to learn from.

As you go to shorter time scales you get more usable data, but then you also need to deal with other issues such as latencies/jitter, market impact, complex order types, order book queues, etc. It becomes a different game.

Re: Using Reinforcement Learning in the Algorithmic Trading Problem

#102

Post by jmeister »

outlace wrote: post #43

Anyone interested in delving into reinforcement learning should check out our book https://www.manning.com/books/deep-reinforcement-learning-in... - we cover actor critic (used in this paper) to multi-agent to relational/attention models

Thanks, looks very useful.

Since you seem to be industry practitioners: I moved away from RL 10 yrs back disillusioned with lack of real-world applicability. Has that changed significantly? The only major name I’ve heard of is Vowpal Wabbit. Maybe there are more applications being done in stealth. Any insight? Thanks

Re: Using Reinforcement Learning in the Algorithmic Trading Problem

#104

Post by anjc »

vecter wrote: post #97
stainforth wrote: post #91

Earlier quoted context omitted.

If the market flips against me, I just double my bet on the next play.

That's a great way to go broke very quickly.

Only if you don't have infinite money on the side, in which case it's guaranteed to be profitable

Re: Using Reinforcement Learning in the Algorithmic Trading Problem

#105

Post by anjc »

1024core wrote: post #4

Every time such a paper like this comes out, I have to ask myself: if they knew how to make money like this, why would they tell anyone?

Because they're academics who choose to forego money in order to advance their career in academia. They could also generally double their income overnight by choosing to move to industry. Many do, most choose not to.

Re: Using Reinforcement Learning in the Algorithmic Trading Problem

#106

Post by mardifoufs »

hasa wrote: post #103

Human greediness is endless. Using AI for stock trading is just unethical and waste of resources. We are sick.

AI has been used in stock trading for decades now. Also, why does it even matter? Why is it unethical to use AI for trading to begin with?

Re: Using Reinforcement Learning in the Algorithmic Trading Problem

#107

Post by anjc »

Many people are criticising their backtest but I don't understand why. Their test data is sequential to their training data, considers time, and doesn't overlap. They can't overfit to their test data. In any other area of ML this would be an acceptable scheme, why is this unacceptable here?

Re: Using Reinforcement Learning in the Algorithmic Trading Problem

#109

Post by dkersten »

stainforth wrote: post #91
young_rutabaga wrote: post #65

Earlier quoted context omitted.

High turnover, high costs. You flip your position too often

If the market flips against me, I just double my bet on the next play.

the market can stay irrational longer than you can stay solvent

Martingale is a sure-fire way to lose all your money rather quickly.

Re: Using Reinforcement Learning in the Algorithmic Trading Problem

#110

Post by johnmoberg »

jmeister wrote: post #102
outlace wrote: post #43

Anyone interested in delving into reinforcement learning should check out our book https://www.manning.com/books/deep-reinforcement-learning-in... - we cover actor critic (used in this paper) to multi-agent to relational/attention models

Thanks, looks very useful. Since you seem to be industry practitioners: I moved away from RL 10 yrs back disillusioned with lack of real-world applicability. Has that changed significantly? The only major name I’ve heard of is Vowpal Wabbit. Maybe there are more applications being done in stealth. Any insight? Thanks

You might be interested in the recently-launched Covariant (https://covariant.ai/), they apparently actually have systems in production. Pieter Abbeel is one of the founders and they have some pretty "heavy" investors, like Jeff Dean, Geoffrey Hinton, and Yann LeCun.
Post reply on HN