Earlier quoted context omitted.
That's a great way to go broke very quickly.
Only if you don't have infinite money on the side, in which case it's guaranteed to be profitable
Martingala with transaction costs are not profitable.
Using Reinforcement Learning in the Algorithmic Trading Problem
111–120 of 139 posts
Re: Using Reinforcement Learning in the Algorithmic Trading Problem
#112Many people are criticising their backtest but I don't understand why. Their test data is sequential to their training data, considers time, and doesn't overlap. They can't overfit to their test data. In any other area of ML this would be an acceptable scheme, why is this unacceptable here?
What people are complaining about is not the overfitting, but the unrealistic assumptions in the backtest. In the real world there is slippage, latencies/jitter, special market open regimes, hidden orders, market impact, front-running, variable fees, and all kinds of other complexities. Their transaction costs are apparently also an unreasonable assumption. Sophisticated simulators used in professional trading firms can account for such things to some extent, but most academic papers conveniently ignore these complexities and just assume they can trade at whatever price the data tells them. It's completely unrealistic.
To answer your original question about overfitting, they can still overfit to test data by running a lot of experiments with different hyperparameters, architectures and parts of the data, and only report what has worked. There are also more complex ways that test data can leak into training data (see the book Advances in financial ML for a good overview). You can already see this is likely the case just from the variance in their results and trades. They also don't compare to baselines. It's not unlikely that the results are just random and they fail to report those experiments that didn't work. Of course, you cannot prove this without having an exact log of all things they ever did to the data. But again, that's not the main issue here.
Re: Using Reinforcement Learning in the Algorithmic Trading Problem
#113Many people are criticising their backtest but I don't understand why. Their test data is sequential to their training data, considers time, and doesn't overlap. They can't overfit to their test data. In any other area of ML this would be an acceptable scheme, why is this unacceptable here?
What people are complaining about is not the overfitting, but the unrealistic assumptions in the backtest. In the real world there is slippage, latencies/jitter, special market open regimes, hidden orders, market impact, front-running, variable fees, and all kinds of other complexities. Their transaction costs are apparently also an unreasonable assumption. Sophisticated simulators used in professional trading firms…
> They also don't compare to baselines
This is certainly worth criticising
> they can still overfit to test data by running a lot of experiments with different hyperparameters, architectures and parts of the data, and only report what has worked.
but this is a different accusation from accidentally overfitting or leaking, i.e. it would mean that they're dishonest and cherrypicked their data in such a way that it hides overfitting and leakage. This criticism can be levelled at every ML paper, but in this case they detail their architecture, provide the code, and provide a Jupyter notebook to let people try it themselves.
> just assume they can trade at whatever price the data tells them. It's completely unrealistic.
I think that this is a fair assumption for highly liquid markets and relatively small trades, and if it's a fair assumption then all of your criticisms (slippage etc) don't apply to the extent that they'll break the approach. Also, if the approach works then trade size (fees aside) and being frontrun also wont apply because presumably large HFT firms can use it.
Overall I think your criticisms are valid, but imo they don't invalidate a promising approach, they're just the next thing to test.
Re: Using Reinforcement Learning in the Algorithmic Trading Problem
#114Earlier quoted context omitted.
As with most industries, it depends. But junior people typically make in the 200-500K range. Then as you gain experience, develop your own ideas/strategies and are able to manage risk appropriately, the sky is the limit. The closer you are to managing money that's being invested the more you make. If you can run a 1.5 - 2 Sharpe strategy and never dip below a ~5% drawdown, i.e. probably have substantial positive skew…
Is 200-500k still true? It used to be, but I think it has decreased significantly over the last decade. I'd say most junior people in this field are making about the same or less than software engineers these days. But like you said, the range here is incredibly wide and largely depends on how well your strategies do and if you have your own desk/fund.
I don’t think it is. I worked for one of the major HFTs and new graduates earned far less than that. In addition the churn rate was high - I’d say most new graduate hires didn’t last more than 2 years and what you learned in those 2 years was often not much use in terms of experience useful for other career paths.
Bonus distribution was reverse exponential. Like traditional consultancy partnerships, a small few of the old hands made serious money but those at the “bottom” made ok money but after a few years their FAANG based contemporaries were doing better. Advancing up the ranks was not guaranteed even if you survived the frequent blood lettings.
Re: Using Reinforcement Learning in the Algorithmic Trading Problem
#115Earlier quoted context omitted.
I actually believe that trading is an interesting problem that should be studied more in Academia and Machine Learning. It has many aspects (sparse rewards, long-time horizons, simulation-to-real-world transfer, non-stationary data distributions, etc) that current ML algorithms struggle with. Unfortunately it seem like most ML people are not really interested in trading, perhaps because it has such a bad reputation (…
i think it’s just because they want to publish, and as you say it’s not easy to find a really good publicly available dataset or simulator. I think if these were publicly available and there were a Python package, people would rapidly get interested in RL for trading. (if it even works for trading — i don’t know much about it but maybe simpler techniques work best, in which case there would be little chance of produc…
I would actually say the opposite - its much easier to get hold of a financial dataset than for other fields. There are packages for easily downloading Yahoo finance data e.g https://github.com/ranaroussi/yfinance
Re: Using Reinforcement Learning in the Algorithmic Trading Problem
#116As someone who has written about this previously [0], worked briefly in HFT before, and read dozens of papers on the subject, I can say with very high confidence that the results are not to be trusted. This paper, just like pretty much any academic paper on the subject, ends with a backtest on historical data, not a real system. Not only is it (very!) easy to overfit backtests (especially with so little data they are…
I also worked in HFT and have no idea what you mean when you say other HFT shops can front-run your orders?
To front run someone’s order you need to have advance information of their order s? Normally this means the front runner is operating as a broker. I can’t imagine any HFTs using other HFTs as brokers to forward their orders to the exchange?
Re: Using Reinforcement Learning in the Algorithmic Trading Problem
#117Re: Using Reinforcement Learning in the Algorithmic Trading Problem
#118Earlier quoted context omitted.
That's only true in the sense of opportunity cost. I may buy something at $10 and sell it at $15 making a $5 profit. Then it may go to $20. Did I lose $5/share? Sure. But in reality I wasn't a "loser". I find that in reality opportunity cost rarely matters.
If you buy something at $10 and sell it at $15, where are the $5 profit coming from? From the other market participants, e.g. someone selling to you for $10 and later buying it back for $15, losing $5 in the process. Your profit and their loss sum to zero, which is what "zero-sum" means. It has absolutely nothing to do with opportunity cost, or whether you, personally, are a "loser". But if you're a "winner", someone…
You didn't take another very important unit into account: time and risk. Let's say you want to sell your used iPhone. You might get $150, if you wait around for the right buyer, but this might take a while and even after waiting it is not guaranteed you find a buyer for that price. The price could even go down to $80. Alternatively you could go to a pawnshop and get $100 dollars instantly. And if you are happy with that price you can go on with your life and focus on other things.
Re: Using Reinforcement Learning in the Algorithmic Trading Problem
#119Earlier quoted context omitted.
There are some ML problems where it is fundamentally impossible to use historical data to make accurate forward looking predictions as its not IID. These fields require you very carefully capture data on sub-optimal choices. In the case of trading this means making explicitly bad trading decisions some portion of the time, and teams that have done this at any scale are unlikely to share the data. In the case of tradi…
I don't get it - if you have accurate historical data, how is this different from having access to current real-time data? Why can't you pretend you live 20 years in the past and use the data you have as if it were real-time?
Fundamentally, the issue is that in real time you may not be able to make the trade that your algorithm chose.
You could get close if you had the actual book prices at any given time, but even then, you might lose to someone who is 1 millisecond faster.
So no, backtesting can simulate reality.
Interactive Brokers offers a simulated account where you can practice "live" trading, although it's still not the same, since there's no money involved. But if I see a paper tested on an IB simulated account I'll be very interested, and it'll be too late already.
Re: Using Reinforcement Learning in the Algorithmic Trading Problem
#120Earlier quoted context omitted.
You make a good point. I've also gotten into trading because I enjoy the algorithmic and mathematical aspects, and I would love to share more of what has been working for me and write extensively about it. And there are probably more people like that out there. However, trading has such a bad reputation and uncertain future that I am not sure that's a good career move. I'm torn. You're right that there are probably s…
So how should we evaluate the quality of a paper on trading AI? I mean the authors might not have access to real data, but their ideas might still be good.
Papers on trading models that show feature importance (rather than backtest results) are more valuable.