Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Using Reinforcement Learning in the Algorithmic Trading Problem

arxiv.org

Post reply on HN
Print view View original

31–40 of 139 posts

Re: Using Reinforcement Learning in the Algorithmic Trading Problem

#31

Post by traderthrow454 »

amiga_500 wrote: post #27
traderthrow454 wrote: post #24

Earlier quoted context omitted.

I think thats an overly simplistic view of things. The market is big and many participants trade at different frequencies. Large pension funds need liquidity to move big blocks of stock for their quarterly and monthly rebalances, and the big medium term statistical arbitrage traders provide liquidity for them to do so. HFT players provide liquidity for the stat arb players. The classes of participants with different…

I knew someone would come in with "liquidity". Many HFT jump out when things get volatile, when liquidity is actually required. Ultimately HFT is doing nothing of societal value, the race down to zero is never-ending and we are wasting huge amounts of resources on a totally pointless march towards zero. Exchanges should introduce random delays to allow market participants who really want to hedge / buy / sell, then w…

My initial comment was discussing speculative trading in general, but since you mostly brought up some common anti-HFT tropes I might as well address them.

> Many HFT jump out when things get volatile, when liquidity is actually required.

Do you have a citation on that? If you look the preliminary Q1 results of Virtu Financial [0] (only publicly traded HFT) they seem to be doing more trading than ever in these volatile markets.

> Ultimately HFT is doing nothing of societal value, the race down to zero is never-ending and we are wasting huge amounts of resources on a totally pointless march towards zero.

HFT is a mature industry. Latencies have mostly stabilized, and profitability is way down in the last few years. Many firms are merging/consolidating. So in the past few years society is actually spending fewer resources - both financially and from a human capital standpoint on HFT than it did in the past.

> Exchanges should introduce random delays to allow market participants who really want to hedge / buy / sell, then we can shift some of the resources to the real world.

IEX is doing something relatively similar to that for a few years now. They have ~3% of US equities market share. People have the option of trading there but they mostly choose not to.

> The system is hugely inefficient. The costs as latencies get lower are ever higher, for an extremely similar end result. The law of diminishing returns.

Due to consolidation, costs are actually decreasing. Could it be that the market is... working?

[0] - https://ir.virtu.com/press-releases/press-release-details/20...

Re: Using Reinforcement Learning in the Algorithmic Trading Problem

#32

Post by conformist »

lordnacho wrote: post #26

Former hedge fund and HFT quant trader here. There's a lot of papers to be found claiming some sort of strategy. I don't want to go to cynicism immediately. But we'll get there: - Trading isn't just about deciding what to buy and sell, the sexy part that everyone thinks is great. I even had colleagues who thought they were special because they worked closer to the strategies, which meant that certain less glamourous…

Yes, this.

The small subsets and super high Sharpe ratios look suspicious.

Further red flags in this particular case:

- Completely unclear what kind of data they're using. Are they assuming they can buy and sell one individual contract at the bid price each minute? Or did I miss some crucial information about bid-ask spreads?

- Abstract mentions a profit, not an information ratio/Sharpe ratio or anything similar.

- During training they need to tweak the reward function in order to not end up with "buy and hold"? How good is their strategy compared to buy and hold?

- Plots without proper labels.

Re: Using Reinforcement Learning in the Algorithmic Trading Problem

#33

Post by traK6Dcm »

amiga_500 wrote: post #11
jawns wrote: post #5

Earlier quoted context omitted.

> people who are actually "good" at trading don't publish papers, they silently make money I've long understood that this was true. It makes intuitive sense. But are there any cases where it is not true? Is it possible to "spread the wealth" when it comes to trading, or any money-making endeavor? Or does it always reduce down to "I win only because you lose"?

> Or does it always reduce down to "I win only because you lose"? It is a zero sum game. Nobody is producing anything, therefore for one to win another must lose.

That's overly simplistic. While the overall system may be zero-system over an infinitely long time horizon, this doesn't typically matter in practice. It can be positive sum for participants over some time horizon they care about.

For example, an HFT trader make pennies from each trade by exploiting tiny price inefficiencies. He essentially takes money from a "stupid" retail investor who does not know how to optimize his trades. However, the retail investor may not actually care about optimizing trades and just wants to liquidate assets or make a long-term (10+ years) bet. He is totally fine with throwing away a few dollars because optimizing his trades through complex algorithms would be too much work. Here, both parties win, the HFT trades gets paid because he provides convenience, or liquidity, to the retail trader. The same would apply to any human market maker, it doesn't have to be HFT.

And yes, HFT liquidity may disappear during HUGE market movements due to risk, but it doesn't disappear as long as both parties get what they want and the risk is manageable, which is "most of the time". Of course, HFT has other issues such as the race to zero and unfair advantages for a few central players, and I don't want to defend HFT. But saying that "it's all zero sum" is not correct.

An analogy is your nearest grocery store. They're a market maker because they buy from the manufacturer and sell to the consumer and profit from the spread. Do you also argue that these are all zero-sum and we should cut them all out and connect all consumers and farmers directly? And their liquidity also disappears when black swans (corona) happens :)

Re: Using Reinforcement Learning in the Algorithmic Trading Problem

#34

Post by amiga_500 »

traderthrow454 wrote: post #31
amiga_500 wrote: post #27

Earlier quoted context omitted.

I knew someone would come in with "liquidity". Many HFT jump out when things get volatile, when liquidity is actually required. Ultimately HFT is doing nothing of societal value, the race down to zero is never-ending and we are wasting huge amounts of resources on a totally pointless march towards zero. Exchanges should introduce random delays to allow market participants who really want to hedge / buy / sell, then w…

My initial comment was discussing speculative trading in general, but since you mostly brought up some common anti-HFT tropes I might as well address them. > Many HFT jump out when things get volatile, when liquidity is actually required. Do you have a citation on that? If you look the preliminary Q1 results of Virtu Financial [0] (only publicly traded HFT) they seem to be doing more trading than ever in these volati…

I'm certain HFT vols are up in the volatility. Not sure that proves or disproves anything.

HFT is hugely expensive to maintain. Staffing and equipment costs are massive.

HFT choose not to trade on exchanges with delays because they want to exploit latency advantages. That's why govt should regulate this on all exchanges to just wipe it out.

Consolidation is not good because it will collapse into a monopoly. Which will require regulation also, because markets do not "work" on their own.

I'll check back later, going for a walk outside now.

----

Here is my reply, despite not replying for 40 minutes, I've put something about capitalism on HN which immediately gets you a fake "you're submitting too fast".

Cannot post any more after that because HN doesn't like discussion unless it's about a tiny window that they agree with. Perhaps someone flagged my comment - if so they make this board a far more boring place to be.

---

HFT != electronic trading. Many firms are automating front-to-back, that is unequivocally a good thing. HFT is latency arb.

Re: Using Reinforcement Learning in the Algorithmic Trading Problem

#35

Post by auntienomen »

cultus wrote: post #10
1024core wrote: post #4

Every time such a paper like this comes out, I have to ask myself: if they knew how to make money like this, why would they tell anyone?

It's a weird subject for academic study in the first place. Trying do do trading strategies that beat the market is one of the few things that really is a zero-sum game. In the absence of independent scientific interest to optimizing these strategies, what's the point? You might as well study how to optimize strategies for ultimate frisbee or something.

It's probably best understood as either job audition, fund-raising, or mathematical entertainment.

Re: Using Reinforcement Learning in the Algorithmic Trading Problem

#36

Post by traderthrow454 »

amiga_500 wrote: post #34
traderthrow454 wrote: post #31

Earlier quoted context omitted.

My initial comment was discussing speculative trading in general, but since you mostly brought up some common anti-HFT tropes I might as well address them. > Many HFT jump out when things get volatile, when liquidity is actually required. Do you have a citation on that? If you look the preliminary Q1 results of Virtu Financial [0] (only publicly traded HFT) they seem to be doing more trading than ever in these volati…

I'm certain HFT vols are up in the volatility. Not sure that proves or disproves anything. HFT is hugely expensive to maintain. Staffing and equipment costs are massive. HFT choose not to trade on exchanges with delays because they want to exploit latency advantages. That's why govt should regulate this on all exchanges to just wipe it out. Consolidation is not good because it will collapse into a monopoly. Which wil…

> I'm certain HFT vols are up in the volatility. Not sure that proves or disproves anything.

You said that HFTs jump out when things get volatile. But then I show you evidence that they trade more in high volatility. Do you not see the contradiction?

> HFT is hugely expensive to maintain. Staffing and equipment costs are massive.

HFT firm equipment costs are a few racks of high end servers and some expensive networking equipment, and a few dozen to few hundred highly paid engineers. Before HFT there were literally _thousands_ of human traders working at different banks doing the same work by hand. Sure in an absolute sense HFT is expensive, but relative to the alternative it is cheap.

> HFT choose not to trade on exchanges with delays because they want to exploit latency advantages. That's why govt should regulate this on all exchanges to just wipe it out.

If HFT is so bad, then the non-HFT market participants should choose to trade at the delay exchanges that make it harder for HFTs. But they choose not to, generally, because the liquidity - and thus trading costs are higher than trading at the non-delay exchanges.

> Consolidation is not good because it will collapse into a monopoly. Which will require regulation also, because markets do not "work" on their own.

First you say that HFT is bad because costs are high, but now you say that falling costs are bad because it will lead to monopoly - do you see the contradiction? For what its worth, Although there are fewer firms there is still a lot of competition among the remaining firms.

Re: Using Reinforcement Learning in the Algorithmic Trading Problem

#37

Post by dchichkov »

idohft wrote: post #8
traK6Dcm wrote: post #2

As someone who has written about this previously [0], worked briefly in HFT before, and read dozens of papers on the subject, I can say with very high confidence that the results are not to be trusted. This paper, just like pretty much any academic paper on the subject, ends with a backtest on historical data, not a real system. Not only is it (very!) easy to overfit backtests (especially with so little data they are…

++ this. If they haven't tested this in actual trades and measured results, it's probably worthless. Even backtested strategies at actual firms observe decays (or don't work) when they get put live. And those are places where they invest in (and are incentivized to get right!) backtesting methodology.

Yes. To add to that, leakage of information is very hard to eliminate during back-testing. Even a fractional bit of information is already too much. In academic papers this is usually ignored.

Re: Using Reinforcement Learning in the Algorithmic Trading Problem

#38

Post by cultus »

auntienomen wrote: post #35
cultus wrote: post #10

Earlier quoted context omitted.

It's a weird subject for academic study in the first place. Trying do do trading strategies that beat the market is one of the few things that really is a zero-sum game. In the absence of independent scientific interest to optimizing these strategies, what's the point? You might as well study how to optimize strategies for ultimate frisbee or something.

It's probably best understood as either job audition, fund-raising, or mathematical entertainment.

Sure, I find mathematical finance absolutely fascinating. I just don't think it's worth putting a lot of research energy into. It's usually the boring stuff that matters.

Re: Using Reinforcement Learning in the Algorithmic Trading Problem

#39

Post by traK6Dcm »

cultus wrote: post #10
1024core wrote: post #4

Every time such a paper like this comes out, I have to ask myself: if they knew how to make money like this, why would they tell anyone?

It's a weird subject for academic study in the first place. Trying do do trading strategies that beat the market is one of the few things that really is a zero-sum game. In the absence of independent scientific interest to optimizing these strategies, what's the point? You might as well study how to optimize strategies for ultimate frisbee or something.

Why do you think it's a weird subject of study, but games like Chess, Go, and Starcraft are not considered weird? Aren't both studied for purely their benchmark potential as opposed to the problem itself? Why are games widely accepted, but trading is "weird"?

Re: Using Reinforcement Learning in the Algorithmic Trading Problem

#40

Post by huac »

traderthrow454 wrote: post #20
traK6Dcm wrote: post #2

As someone who has written about this previously [0], worked briefly in HFT before, and read dozens of papers on the subject, I can say with very high confidence that the results are not to be trusted. This paper, just like pretty much any academic paper on the subject, ends with a backtest on historical data, not a real system. Not only is it (very!) easy to overfit backtests (especially with so little data they are…

If you want to read useful academic papers about trading there is one author in particular who is actually not bad - Zura Kakushadze. Most of his stuff is applicable to mid-frequency trading, not HFT. He worked at WorldQuant (reputable trading firm) and the founder of WQ, Igor Tulchinsky, is a coauthor on one of his papers. Example of a pretty interesting and accessible one - is "101 Formulaic Alphas" [0]. [0] - http…

picking a random one out of the pile:

> Alpha#90: ((rank((close - ts_max(close, 4.66719)))^Ts_Rank(correlation(IndNeutralize(adv40, IndClass.subindustry), low, 5.38375), 3.21856)) * -1)

I wonder how these magic numbers get picked (4.66719, 5.38375, etc) -- I guess there is some optimization solver which attempts to find the most profitable variables for a given alpha formulation, but isn't this approach also very vulnerable to overfit?

Post reply on HN