Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Using Reinforcement Learning in the Algorithmic Trading Problem

arxiv.org

Post reply on HN
Print view View original

81–90 of 139 posts

Re: Using Reinforcement Learning in the Algorithmic Trading Problem

#81

Post by godelmachine »

smdz wrote: post #46
traK6Dcm wrote: post #2

As someone who has written about this previously [0], worked briefly in HFT before, and read dozens of papers on the subject, I can say with very high confidence that the results are not to be trusted. This paper, just like pretty much any academic paper on the subject, ends with a backtest on historical data, not a real system. Not only is it (very!) easy to overfit backtests (especially with so little data they are…

> In addition to that, people who are actually "good" at trading don't publish papers, they silently make money. Well, that is mostly true. But never discount anything. There are people like me who used to love the data analysis and prediction part in these markets. I got hooked to the markets because of it. I was not interested in making money and naively thought my average pay was good enough. When I first built (o…

Would you kindly give me a fair idea as to what’s a good amount of money to be made in this field?

Re: Using Reinforcement Learning in the Algorithmic Trading Problem

#82

Post by deehouie »

traK6Dcm wrote: post #19
deehouie wrote: post #14

Earlier quoted context omitted.

While it's so easy to dismiss someone's work as flawed (sure, backtest is illusional but do you have anything better?), which I think it may be, I always read it and try to understand what they're up to. Sure, academic folks may have no clue about market microstructure and other complexities, but if they could solve, or make some way toward solving the difficult problems in stochastic processes, they're already worth…

I actually believe that trading is an interesting problem that should be studied more in Academia and Machine Learning. It has many aspects (sparse rewards, long-time horizons, simulation-to-real-world transfer, non-stationary data distributions, etc) that current ML algorithms struggle with. Unfortunately it seem like most ML people are not really interested in trading, perhaps because it has such a bad reputation (…

"most ML people are not really interested in trading"

You couldn't be more wrong on this. Stock market trading has the lowest barrier to entry of any endeavor. All you need is $1000 and Robinhood account, which you can open one on your phone in 5 min or less.

I've been following HN for a while, every time someone comes up with a trading algo or posts a link to algo, there's were hundreds of upvotes, lots of comments.

Re: Using Reinforcement Learning in the Algorithmic Trading Problem

#83

Post by samfisher83 »

1024core wrote: post #4

Every time such a paper like this comes out, I have to ask myself: if they knew how to make money like this, why would they tell anyone?

Sometimes you come up with a strategy but you don't have bankroll or infrastructures to execute it. It might help you get a job. Maybe the strategy works for a short time. If someone wants to pay you 400k to write algorithms that might be better option.

Re: Using Reinforcement Learning in the Algorithmic Trading Problem

#85

Post by fakedang »

solidasparagus wrote: post #30
traK6Dcm wrote: post #25

Earlier quoted context omitted.

I don't claim that all the science that isn't actively being applied yet is BS, but this kind of science typically happens within trading firms, tested on real-world data, and is not being published on arXiv. As a side note, what this specific paper here did is neither novel not innovative, so it's very fair to criticize it. A3C is 4 years old, and they just take it and run it on some data. It's like downloading a co…

I was referring to these statements: > This paper, just like pretty much any academic paper on the subject, ends with a backtest on historical data, not a real system > Whenever you see a paper ending with a backtest you can already assume it's BS.

I guess he was referring to the fact that though the people may be great, their published content in that one paper need not be so.

Re: Using Reinforcement Learning in the Algorithmic Trading Problem

#86

Post by deehouie »

currymj wrote: post #28
traK6Dcm wrote: post #19

Earlier quoted context omitted.

I actually believe that trading is an interesting problem that should be studied more in Academia and Machine Learning. It has many aspects (sparse rewards, long-time horizons, simulation-to-real-world transfer, non-stationary data distributions, etc) that current ML algorithms struggle with. Unfortunately it seem like most ML people are not really interested in trading, perhaps because it has such a bad reputation (…

i think it’s just because they want to publish, and as you say it’s not easy to find a really good publicly available dataset or simulator. I think if these were publicly available and there were a Python package, people would rapidly get interested in RL for trading. (if it even works for trading — i don’t know much about it but maybe simpler techniques work best, in which case there would be little chance of produc…

Ther are plenty of free datasets out there. You can get upward of 10 yrs of daily OHLC stock data on yahoo finance. The amazing thing is yf has S&P 500 index since 1927. Free!

Quandl has many free, or low cost stock market/commodity datasets.

I'm not sure what you mean by a "simulator". One of the greatest challenge applying RL to stock mkt is precisely that the market itself is not a MDP.

Re: Using Reinforcement Learning in the Algorithmic Trading Problem

#87

Post by johndoe42377 »

This is just an attention whoring, bragging and self-praise, lots of ambitions with little or no real education. Canonical definition of a russian.

Any attempt to apply mathematical methods for modeling of fully observable, deterministic, stable systems, like probability or neural networks (they assume, by definition, stable patterns, which arise from underlying laws at the training stage and in feedback loops) will inevitably lead to a failure. Always. In principle.

Gamefication of markets is even more stupid idea, hence, again, an adequate game model requires a well-defined set of rules which does not change in the middle of the game, and, like any other model or simulation, requires that all (every) relevant factors has been adequately represented, which will never be the case.

Last but not least, one never trade in the middle of the chart, on the past data. If a model fits the past data it only means that it has been trained on the past data, which is already "stable". It will be unable to deal with new data because it will be nothing like training and testing set in principle.

Even classic methods from signal processing will not do, because a signal implies regularly and predictability (a predetermined set of probabilities), while markets do not possess such properties.

This is just misapplication of methods without any understand of underlying phenomena, methods being used, their applicability and limitations.

Read some Hamming lmao.

Re: Using Reinforcement Learning in the Algorithmic Trading Problem

#88

Post by traderthrow454 »

amiga_500 wrote: post #58
twic wrote: post #52

Earlier quoted context omitted.

> You said that HFTs jump out when things get volatile. But then I show you evidence that they trade more in high volatility. Do you not see the contradiction? We should distinguish between volatility at different timescales. An HFT might well be very active over a volatile month, but may still turn off over a very volatile second. I've always assumed that the "HFTs jump out during volatility" complaint was about the…

Exactly, that was my point and I thought that would be understood by someone in the domain. Same for HFT vs "electronic" in general, again someone in the domain would get that distinction immediately.

HFT = electronic trading, or at least the bulk of it. The human traders that existed before HFT firms & HFT desks at banks were largely doing the same thing that HFT firms do now - making markets on a wide range of securities, but less efficiently.

Latency-arb is a small part of HFT, if you consider strategies that provide liquidity to be "good HFT" and strategies that take liquidity (via latency arb or other arbs) to be "bad HFT" then those strategies are largely executed by the same market participants.

Re: Using Reinforcement Learning in the Algorithmic Trading Problem

#89

Post by imcoconut »

godelmachine wrote: post #81
smdz wrote: post #46

Earlier quoted context omitted.

> In addition to that, people who are actually "good" at trading don't publish papers, they silently make money. Well, that is mostly true. But never discount anything. There are people like me who used to love the data analysis and prediction part in these markets. I got hooked to the markets because of it. I was not interested in making money and naively thought my average pay was good enough. When I first built (o…

Would you kindly give me a fair idea as to what’s a good amount of money to be made in this field?

As with most industries, it depends. But junior people typically make in the 200-500K range. Then as you gain experience, develop your own ideas/strategies and are able to manage risk appropriately, the sky is the limit. The closer you are to managing money that's being invested the more you make. If you can run a 1.5 - 2 Sharpe strategy and never dip below a ~5% drawdown, i.e. probably have substantial positive skew in returns, you can make in the millions or tens of millions at the right fund. Note that as the OP correctly alluded to, this is much more difficult to do live than in a backtest.

Re: Using Reinforcement Learning in the Algorithmic Trading Problem

#90

Post by p1esk »

lumost wrote: post #80
p1esk wrote: post #68

Earlier quoted context omitted.

So how should we evaluate the quality of a paper on trading AI? I mean the authors might not have access to real data, but their ideas might still be good.

There are some ML problems where it is fundamentally impossible to use historical data to make accurate forward looking predictions as its not IID. These fields require you very carefully capture data on sub-optimal choices. In the case of trading this means making explicitly bad trading decisions some portion of the time, and teams that have done this at any scale are unlikely to share the data. In the case of tradi…

I don't get it - if you have accurate historical data, how is this different from having access to current real-time data? Why can't you pretend you live 20 years in the past and use the data you have as if it were real-time?
Post reply on HN