Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Lessons learned building an ML trading system

tradientblog.com

Post reply on HN
Print view View original

101–110 of 160 posts

Re: Lessons learned building an ML trading system

#101

Post by nsainsbury »

thundergolfer wrote: post #76

This post is an exemplar of the crucial relationship between domain-specific knowledge and ML competency in the ML space. The bulk of the post is detailing the tricky ins and outs of trading, and overall the author gives the impression that they're broadly knowledgeable about stock markets. Contrast this post with those you see with ML hobbyists who delve into medicine or fake-news and produce useless results testame…

Bingo. I've made this comment several times on Hacker News in the past as well and in my opinion it's the number one reason I've seen ML projects fail to have impact at companies it's deployed: the operators (typically higher level Math/CS types) simply don't understand the domains well enough and so frequently end up making absurd recommendations/suggestions (often to the detriment of other business areas).

The successful application of ML requires a deep understanding of the domain it's being applied in.

Re: Lessons learned building an ML trading system

#102

Post by arthurcolle »

anigbrowl wrote: post #32
arthurcolle wrote: post #17

Earlier quoted context omitted.

Don’t forget price discovery.

But you're not discovering anything valuable because there may not be a person on the other end of a trade. You're just learning about the chaotic boundaries of the trading algorithm.

No, your interpretation is incorrect, and that's not how this works at all.

> There may not be a person on the other end of the trade

uh... what? If the executed order fills, then there was someone on the other side of the trade.

Anyways, by providing liquidity in any market as a market maker, you are effectively aiding in the overall process of price discovery. HFT shops are generally market makers, although other strategies are also possible depending on how the operation intends to generate alpha. For HFT MMs, they pretty much only make money by clipping spread, i.e. submitting dual buy and sell orders at the midprice with the expectation that they will make the (ask-bid)/2 on average. They then cancel these orders as the orderbook's structure changes and as prices and markets move, resubmitting at "better" (more favorable) levels.

It doesn't matter if people are buying or selling or both. HFT MMs provide a valuable service to financial market participants - if participating counterparties submit orders and cross a HFTs latest uncancelled order, it will fill, allowing market participants to quickly gain or lose exposure to their security or instrument of choice.

There's no magic here and you seem really confused about the underlying dynamics of trading and market microstructure.

Re: Lessons learned building an ML trading system

#103

Post by Akababa »

perl4ever wrote: post #89
growse wrote: post #85

Earlier quoted context omitted.

> The substance of the complaint is the speed of the auction that discovers the price. The faster price discovery is being done, the fewer traders can participate. The faster the price discovery, the shorter the validity time. This means a more accurate price. > If a few milliseconds can change the value of a financial instrument then we have to accept that waiting a second or two will allow a lot more traders to ree…

"The faster the price discovery, the shorter the validity time. This means a more accurate price." A continuous market price is an illusion; let's not forget that. Every market is made up of trades, which are discrete, and every price has an unstated amount of uncertainty, which may be large by any standards. You can't know from first principles whether a change in quoted price is even a change in the market, because…

I don't believe liquidity necessarily has anything to do with efficiency, but more liquidity can never hurt. Even if the price of every stock right now is 100x what it should be, I still want lower spreads.

And to address your concerns about depth, I believe that's where market makers (which are related to HFT) come in.

Re: Lessons learned building an ML trading system

#105

Post by Akababa »

roenxi wrote: post #81
jstanley wrote: post #66

Earlier quoted context omitted.

If exchanges added a random delay, that would mean traders have to plan for a longer time horizon, which means they need to offer more conservative prices. That means when you're selling, you get less money, and when you're buying, you get less stuff. Some people have this notion that if their trade got matched against a HFT that they have somehow lost out. The exact opposite is true. If the party on the other side o…

Hang on, that is conflating separate ideas. The substance of your post is describing price discovery - price discovery isn't the issue here. The issue is that the price discovery is being done in milliseconds. The substance of the complaint is the speed of the auction that discovers the price. The faster price discovery is being done, the fewer traders can participate. If a few milliseconds can change the value of a…

> If a few milliseconds can change the value of a financial instrument then we have to accept that waiting a second or two will allow a lot more traders to reevaluate and offer a fair price.

I'm not quite getting this point. If I have a sell order at price x and it's filled by a HFT 1 second before someone with a slower algorithm, how does that result in a more "fair" price for me? If the HFT instead posts a buy order at an unfair price x-1, there's nothing stopping the slower traders from taking my sell order at x one second later.

Re: Lessons learned building an ML trading system

#106

Post by kmangutov »

smeeth wrote: post #100
megaframe wrote: post #40

Earlier quoted context omitted.

Ran an ML model years ago had a number of great months then out of no where no trade I or the ML would make, would work. Looked like someone was front running my orders and messing with my trades. Weird delays, trades would take to long to go through all, and all sorts of odd events on Level 2. Ended up shutting it down took a good 2 months before my manual trades started going through at a normal rate again. This is…

I think, though I'm not sure, that RH makes their money by investing in treasury bills with the cash balances of people's portfolios. You're (most likely) not getting secretly screwed.

Robinhood sells order flow https://www.cnbc.com/2019/04/18/a-controversial-part-of-robi...

Re: Lessons learned building an ML trading system

#107

Post by traK6Dcm »

__d wrote: post #97
traK6Dcm wrote: post #54

Earlier quoted context omitted.

Author here. Perhaps if you already have existing HFT infrastructure and connections to efficiently trade on such exchanges. But such infra costs millions. If you don't have this, you're probably at too large of a disadvantage to find any alpha. At least that's my understanding based on conversations I've had, I've never traded equities.

In a little more detail ... To be competitive in US equities HFT, you need an FGPA with 40GbE ports hosted in a server (which needs to power and cool the FPGA, and deal with the less latency-sensitive bits of your system). You'll need some storage as well. That server needs to be co-located with your target exchange(s) matching engines, and connected via 40GbE. You might additionally want remote market data via mm-wa…

Thanks a lot for the detail. I've always wondered what the actual costs look like.

Re: Lessons learned building an ML trading system

#108

Post by elamje »

smeeth wrote: post #100
megaframe wrote: post #40

Earlier quoted context omitted.

Ran an ML model years ago had a number of great months then out of no where no trade I or the ML would make, would work. Looked like someone was front running my orders and messing with my trades. Weird delays, trades would take to long to go through all, and all sorts of odd events on Level 2. Ended up shutting it down took a good 2 months before my manual trades started going through at a normal rate again. This is…

I think, though I'm not sure, that RH makes their money by investing in treasury bills with the cash balances of people's portfolios. You're (most likely) not getting secretly screwed.

Contrary to popular belief robinhood makes a good bit of money from selling order data, more than most brokerages. This is due to the fact that most robinhood users are inexperienced, and thus, their trades are less likely to indicate big movements of institutional money. I.e. HFTs pay more for data that is less risky.

Edit: Fun fact, the founders of Robinhood worked at an HFT before starting robinhood, which used to be shown on their LinkedIn until several years ago. Bad optics I suppose.

Re: Lessons learned building an ML trading system

#109

Post by mrpopo »

traK6Dcm wrote: post #61
cco wrote: post #9

And in the end, what value was created? "Liquidity in the BTC market"?

Author here. Actually, the system is mostly taking liquidity from the market so it's not even doing that :) Perhaps, "opportunity for others to create more liquidity" Jokes aside, it's actually something I am thinking about a lot. Such systems don't create value, but they extremely intellectually interesting and I've learned a lot. You can say the same for many other projects, for example academic research in many fi…

> But it's intellectually interesting, so people work on it.

No. It's a financially sound way to use time, so people work on it. Anything can be intellectually interesting.

What kind of academic research does not create value? And if so, maybe it deserves to be criticized the same way.

Re: Lessons learned building an ML trading system

#110

Post by jugg1es »

It must be said that it is a lot easier to make money in a stock market that has had low volatility and no significant, prolonged dip in the last 5+ years. My own long-term investments have earned 20% return over the last 5 years with zero trading. I realize that this article is specifically about crypto, but trends in all markets is generally up across the board.
Post reply on HN