Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Lessons learned building an ML trading system

tradientblog.com

Post reply on HN
Print view View original

91–100 of 160 posts

Re: Lessons learned building an ML trading system

#91

Post by perl4ever »

traK6Dcm wrote: post #68
criddell wrote: post #60

Earlier quoted context omitted.

What's the value of HFT? If exchanges were required to add a random delay to very trade to work against high frequency traders, would anything of value be lost?

Many HFT systems provide liquidity. If there is no liquidity, retail investors like you or me cannot buy or sell. Just imagine you want to exchange a currency because you go traveling and the exchange tells you "Sorry, nothing available right now, gotta come back in a few weeks". That's what would happen if there is no liquidity.

We obviously can have liquidity on in-between timescales, because we had it before modern HFT trading, right?

The average person may be overly paranoid about HFT, but it doesn't make sense to say they benefit from it, because they are not going to be in a position where they benefit from an execution in a fraction of a second.

Price improvement of fractions of a penny has gotten silly too. It's easy to think of it as more significant than it is, until you figure it as a percentage (or the spread for that matter).

It's kind of like how ultra-sensitive people are to gas prices...

Re: Lessons learned building an ML trading system

#92

Post by Akababa »

onlyrealcuzzo wrote: post #69
Akababa wrote: post #41

Earlier quoted context omitted.

Through hypothesis testing you can estimate the probability that this was due to luck is very low. Assuming that a monkey would have a 50% chance of profiting on a day, the chance of going a month without a losing day is less than 1 in a billion.

Wouldn't it be better to look at how much this deviates from average market returns during the time frame?

Depends on which null hypothesis you're testing. He told us that he rarely had a down day, and based on Bitcoin's volatility I'd say it's reasonable to assume the probability of it going up or down on any given day is a coin toss (regardless of the longer-term trend and independent of other days). Thus the probability that he's a day-trading monkey is 0.5^n, where n is the number of up days he had in a row. (since he had a few down days it's actually a binomial coefficient but the value is On the other hand, if we wanted to test his 3900% yearly return, we might assume that monkey returns are equal in distribution to Bitcoin's price and then test the hypothesis that he's a monkey via something like a paired t-test. The problem here is that we only have one data point so p-value is undefined, and due to high variance it would probably take about n=10 points to get something significant. The upside of this approach is that you can get a confidence interval for how much better he is than a monkey, instead of just a yes/no answer.

In any case, since the author has at least 365 data points, he probably has an extremely good idea of both a) whether he's a monkey, and b) how much better he is than a monkey.

Re: Lessons learned building an ML trading system

#93

Post by musicale »

criddell wrote: post #60
nickreese wrote: post #11

After having spent an insane amount of time in late 2017/2018 building an HFT bot for Binance I can say this is a pretty solid article. In our case we were doing triangle trading between BTC/ETH/USDT pairs and had our buys/sell delay down to 3-7ms. At one point moving 0.3-0.7% of Binance’s daily volume. Few notes: * Finding an objective point of truth for value when all of the currencies are floating is hard but vita…

What's the value of HFT? If exchanges were required to add a random delay to very trade to work against high frequency traders, would anything of value be lost?

Not really. The real-world value of Google (or any other company) doesn't change meaningfully on a millisecond basis.

Re: Lessons learned building an ML trading system

#94

Post by rb808 »

criddell wrote: post #60
nickreese wrote: post #11

After having spent an insane amount of time in late 2017/2018 building an HFT bot for Binance I can say this is a pretty solid article. In our case we were doing triangle trading between BTC/ETH/USDT pairs and had our buys/sell delay down to 3-7ms. At one point moving 0.3-0.7% of Binance’s daily volume. Few notes: * Finding an objective point of truth for value when all of the currencies are floating is hard but vita…

What's the value of HFT? If exchanges were required to add a random delay to very trade to work against high frequency traders, would anything of value be lost?

The liquidity from bots has replaced the human brokers. Sell side stock trader is a job that mostly no longer exists Eg https://www.bloomberg.com/news/articles/2018-04-30/goldman-t...

Re: Lessons learned building an ML trading system

#95

Post by traK6Dcm »

fny wrote: post #78

Can someone comment on how taxes are handled when automated trades are made like this? It's something that seems wholly absent from the cost calculations.

Author here. Personally, I just don't. I tried doing it but it was too complicated. So I end up just hiring a tax accountant specializing in crypto, send them all the data I have, and pay a few $k. In case something goes wrong, it's their fault and they take the risk.

Re: Lessons learned building an ML trading system

#96

Post by gnaritas »

roenxi wrote: post #82
vecter wrote: post #79

Earlier quoted context omitted.

That's not how things work. You would see much less demand/supply for a particular stock at every price level, which means you'd pay much more to buy less and would have to pay much more to sell less also. There are no upsides to having fewer market makers.

> you would see much less demand/supply Why? If the HFT firm was willing to offer me $X 2 milliseconds ago they are probably still willing to offer $X now. It isn't like there has been time for anything to change; there are going to be short periods of time where there is literally no new information. And they are just as likely to be offering me more now than less as conditions change.

> It isn't like there has been time for anything to change

Wrong, other buyers or sellers could have put in orders in that time changing the price.

Re: Lessons learned building an ML trading system

#97

Post by __d »

traK6Dcm wrote: post #54
echelon wrote: post #20

Can this same strategy be leveraged on zero-fee stock exchanges? Why is crypto the target here?

Author here. Perhaps if you already have existing HFT infrastructure and connections to efficiently trade on such exchanges. But such infra costs millions. If you don't have this, you're probably at too large of a disadvantage to find any alpha. At least that's my understanding based on conversations I've had, I've never traded equities.

In a little more detail ...

To be competitive in US equities HFT, you need an FGPA with 40GbE ports hosted in a server (which needs to power and cool the FPGA, and deal with the less latency-sensitive bits of your system). You'll need some storage as well.

That server needs to be co-located with your target exchange(s) matching engines, and connected via 40GbE. You might additionally want remote market data via mm-wave microwave.

You can probably put together a basic but competitive hardware setup for $70k or so, if you ignore redundancy, and you only need to trade a single market. More realistically, you'll need at least two, plus shared storage, and probably more depending on what markets you intend to trade on.

Then you have monthly costs: colocation for the server(s) ($5k-ish+), port fees for the order entry ($500-ish), port fees for market data ($500-ish), physical connectivity fees ($20k-ish) , cross connect fees for the connectivity ($500-ish), wireless connectivity fees, you might need roof access (more fees), market data fees (per exchange), memberships, and trading costs.

I haven't done this for a while, but it easily adds up to $100k per month or more.

So you need to be making quite a bit to pay off your infra, before you start thinking about profit. And your model will age pretty fast, so you'll want to be working on a few possible replacements concurrently.

It's a tough business.

Re: Lessons learned building an ML trading system

#98

Post by wellactually »

akra wrote: post #80
traK6Dcm wrote: post #68

Earlier quoted context omitted.

Many HFT systems provide liquidity. If there is no liquidity, retail investors like you or me cannot buy or sell. Just imagine you want to exchange a currency because you go traveling and the exchange tells you "Sorry, nothing available right now, gotta come back in a few weeks". That's what would happen if there is no liquidity.

I always had questions about the liquidity on offer though seeing participants flee the market in black swan events. HFT liquidity could be illusionary - its only there when its not quite required similar to how the bank "only offers you a loan when you don't need one". Of course this is exactly the point when liquidity is required; normally there's sufficient liquidity in normal times from market participants. Its e…

Offering liquidity is not equivalent to offering free money. The offer comes at a cost.

Re: Lessons learned building an ML trading system

#99

Post by ScottBurson »

traK6Dcm wrote: post #61
cco wrote: post #9

And in the end, what value was created? "Liquidity in the BTC market"?

Author here. Actually, the system is mostly taking liquidity from the market so it's not even doing that :) Perhaps, "opportunity for others to create more liquidity" Jokes aside, it's actually something I am thinking about a lot. Such systems don't create value, but they extremely intellectually interesting and I've learned a lot. You can say the same for many other projects, for example academic research in many fi…

I think of it as like competitive sport. The value you're providing is the opportunity for others to test themselves against you.

Re: Lessons learned building an ML trading system

#100

Post by smeeth »

megaframe wrote: post #40
nickreese wrote: post #11

After having spent an insane amount of time in late 2017/2018 building an HFT bot for Binance I can say this is a pretty solid article. In our case we were doing triangle trading between BTC/ETH/USDT pairs and had our buys/sell delay down to 3-7ms. At one point moving 0.3-0.7% of Binance’s daily volume. Few notes: * Finding an objective point of truth for value when all of the currencies are floating is hard but vita…

Ran an ML model years ago had a number of great months then out of no where no trade I or the ML would make, would work. Looked like someone was front running my orders and messing with my trades. Weird delays, trades would take to long to go through all, and all sorts of odd events on Level 2. Ended up shutting it down took a good 2 months before my manual trades started going through at a normal rate again. This is…

I think, though I'm not sure, that RH makes their money by investing in treasury bills with the cash balances of people's portfolios. You're (most likely) not getting secretly screwed.
Post reply on HN