Thoughts on low latency trading if exchanges went full cloud • Hacker News Board

Re: Thoughts on low latency trading if exchanges went full cloud

#131

Post by hinkley » Tue, Apr 16, 2024, 5:05 PM UTC

amluto wrote: post #85

posnet wrote: post #54

The biggest current limitation with cloud providers when it comes to exchange tech is the lack of real multicast support. It is rare outside of exchanges, but extremely low latency L1 multicast market data has become the backbone of exchanges, both for fairness and for scalability. Knowing you can saturate your entire network with 10G traffic and every participant will get the same market data packets at the same tim…

> Knowing you can saturate your entire network with 10G traffic and every participant will get the same market data packets at the same time[0] Hold on a second. Multicast is nifty, but it does not perform miracles. If you operate a 10G multicast network and actually saturate it, you will experience drops and buffering-induced delays. Perhaps you can play games with time-synchronous networking, but as far as I know t…

People get weird about the word “saturate”. I think GP is switching to “max sustainable” and expecting everyone else to come along for the ride.

Queuing theory has many many bad things to say about actual saturation.

Re: Thoughts on low latency trading if exchanges went full cloud

#132

Post by jeffreyrogers » Tue, Apr 16, 2024, 5:06 PM UTC

kasey_junk wrote: post #41

jstsch wrote: post #10

Nice article. Wondering though why trading is not done in discrete batches, e.g. 5 second intervals? Trades in the same interval get filled equally or stochastically? Info about trades with that same 5 second batch delay? Is there some (theoretical) market efficiency thing at play? All this HFT feels wasteful and bad for 'regular' human investors.

How do you tie break? If there are more sellers than buyers (or vice versa) at the clearing price?

They already do this for the opening and closing auctions. You can have a market-on-close order or limit-on-close order for example. The market on close orders are guaranteed to fill. The limit orders are filled using price-time priority, so best prices submitted earliest fill first, after the market price orders.

I guess it is possible that there are remaining marketable orders that never fill because of an imbalance one way or the other, but I doubt that ever happens in practice.

Re: Thoughts on low latency trading if exchanges went full cloud

#133

Post by hinkley » Tue, Apr 16, 2024, 5:07 PM UTC

HALtheWise wrote: post #120

amluto wrote: post #85

Earlier quoted context omitted.

> Knowing you can saturate your entire network with 10G traffic and every participant will get the same market data packets at the same time[0] Hold on a second. Multicast is nifty, but it does not perform miracles. If you operate a 10G multicast network and actually saturate it, you will experience drops and buffering-induced delays. Perhaps you can play games with time-synchronous networking, but as far as I know t…

I'm curious if you know what, at a switch level, would actually cause drops and buffering for a 1:N (near-) saturated multicast flow. If all the packets are coming from the same source machine at (perhaps) 9.9Gbps and flowing into the switch, I would expect the switch to robustly redirect all that data with near-zero latency or packet drops to all its output ports. I don't think 10G Ethernet has "backpressure" in a w…

Periodic background traffic like DHCP and background noise causing packet loss.

You can’t run a queue at 100% and have any expectations of latency. In fact the rule of thumb from queueing theory is 50% to avoid latency spikes.

Re: Thoughts on low latency trading if exchanges went full cloud

#134

Post by hinkley » Tue, Apr 16, 2024, 5:14 PM UTC

amluto wrote: post #86

pclmulqdq wrote: post #66

Earlier quoted context omitted.

Pub/sub systems in unicast-only environments are very complex distributed systems to handle the load involved in fan-out routing while maintaining a global order. I had an interviewer once get annoyed with me for suggesting using multicast to solve the fan-out part of a pub/sub system, which made the global ordering part small and simple. We lost a lot by thinking of HTTP as the one true level of network abstraction.

A reliable multicast network that preserves global order even during maintenance and doesn’t drop packets is not something you will find off the shelf. A reliable multi-tenant multicast network also appears to be a rare beast. I’ve only heard of it in finance, and that’s only because it’s private and expensive and all the participants need to be generally nice to each other because it’s a repeated game and the operat…

There was a time in the 00’s where a lot of server hardware had 3 NICs and you could use those for redundancy but a better use was to create three networks: inbound, service and database calls, and administrative.

You had more control over your services talking to each other and control plane tech, thus could make some more guarantees than with inbound data. Don’t cross the streams.

Re: Thoughts on low latency trading if exchanges went full cloud

#135

Post by Kranar » Tue, Apr 16, 2024, 5:22 PM UTC

lucianbr wrote: post #127

Kranar wrote: post #121

Earlier quoted context omitted.

Of course the Average Joe has disadvantages compared to a hedge fund or people who are experts in a field and spend the bulk of their life dedicated to some aspect that the Average Joe is not dedicated towards. If Average Joe wants returns comparable to these hedge funds, then they should stop trying to time to market and instead stick to diversified ETFs and stop worrying about millisecond differences in the stock m…

I mean, you could also write "of course whoever has servers closer to the exchange can do HFT better". "Of course companies that invested a lot in having servers closer will reap the advantages." No, expertise is not the difference. If you're a private person with 100 years experience in trading, you still can't do HFT. You need to be an instutition, have lots of capital to invest in servers, software development mai…

>of course whoever has servers closer to the exchange can do HFT better

Can do better at what? Can get their trade in the order book faster? Yes they can. But does that automatically mean they will make more money? No it does not.

>If you're a private person with 100 years experience in trading, you still can't do HFT.

Of course not, 100 years ago there were no computers. Having 100 years of experience in trading on the pit would not give you any expertise in software development.

Someone with 10 thousand years of experience plowing can't compete against someone with a tractor. That's kind of the point of the tractor...

I'm sorry that it disturbs you that the Average Joe sitting at home with his discount online brokerage account is unable to gain the same kind of benefits putting out individual orders here and there on speculative stocks that he likely knows nothing about, that hedge funds, institutions, and other highly specialized and skilled professionals are able to gain by doing this for a living.

The Average Joe does have access to highly diversified and low fee ETFs, and as I said the Average Joe can reap almost all of the rewards that the best hedge funds and banks do by sticking to those instead of trying to play the market.

Re: Thoughts on low latency trading if exchanges went full cloud

#136

Post by hackerlight » Tue, Apr 16, 2024, 6:38 PM UTC

dusted wrote: post #56

Showing that I don't understand economics while also telling that I don't understand economics: It would probably do the world more good to tweak the structures making ULL trading profitable anyway, it's not like the trading in and of itself brings any value to the broader world, while consuming enormous amounts of resources that could have been spent on actually improving systems that create real value.

Riddle me this. If you got what you wanted, and these value destroying people went away, what would they be replaced with?

Re: Thoughts on low latency trading if exchanges went full cloud

#137

Post by bitcharmer » Tue, Apr 16, 2024, 6:41 PM UTC

amluto wrote: post #85

posnet wrote: post #54

The biggest current limitation with cloud providers when it comes to exchange tech is the lack of real multicast support. It is rare outside of exchanges, but extremely low latency L1 multicast market data has become the backbone of exchanges, both for fairness and for scalability. Knowing you can saturate your entire network with 10G traffic and every participant will get the same market data packets at the same tim…

> Knowing you can saturate your entire network with 10G traffic and every participant will get the same market data packets at the same time[0] Hold on a second. Multicast is nifty, but it does not perform miracles. If you operate a 10G multicast network and actually saturate it, you will experience drops and buffering-induced delays. Perhaps you can play games with time-synchronous networking, but as far as I know t…

Almost all exchanges disseminate market data over multicast these days. If you miss a tick it doesn't matter because by the time a tcp retransmission completes this is old, useless data.

Re: Thoughts on low latency trading if exchanges went full cloud

#138

Post by seanhunter » Tue, Apr 16, 2024, 6:42 PM UTC

amluto wrote: post #85

posnet wrote: post #54

The biggest current limitation with cloud providers when it comes to exchange tech is the lack of real multicast support. It is rare outside of exchanges, but extremely low latency L1 multicast market data has become the backbone of exchanges, both for fairness and for scalability. Knowing you can saturate your entire network with 10G traffic and every participant will get the same market data packets at the same tim…

> Knowing you can saturate your entire network with 10G traffic and every participant will get the same market data packets at the same time[0] Hold on a second. Multicast is nifty, but it does not perform miracles. If you operate a 10G multicast network and actually saturate it, you will experience drops and buffering-induced delays. Perhaps you can play games with time-synchronous networking, but as far as I know t…

Generally speaking how multicast is used in trading situations[1] is you have two networks. On the primary network you do most of your normal IP traffic between applications etc. Then you have a seperate marketdata network that has most of the multicast traffic and it's exclusively used for marketdata. Marketdata generally is delivered on an "As fast as possible" basis[2]. So you don't care too much about occasional drops although fewer is obviously better.

[1] At least in my time in the front office.

[2] For example a very common pattern at the very low level for a marketdata subscription is when you subscribe to marketdata for some symbol the system will actually have a double buffer where it writes into one slot and you read from another slot and every time you read it switches the slots around. This means you can generally accept marketdata as fast as it arrives and process it when you can and you will always get the most recent packet when you ask for the next packet.

Re: Thoughts on low latency trading if exchanges went full cloud

#139

Post by bitcharmer » Tue, Apr 16, 2024, 6:43 PM UTC

HALtheWise wrote: post #120

amluto wrote: post #85

Earlier quoted context omitted.

> Knowing you can saturate your entire network with 10G traffic and every participant will get the same market data packets at the same time[0] Hold on a second. Multicast is nifty, but it does not perform miracles. If you operate a 10G multicast network and actually saturate it, you will experience drops and buffering-induced delays. Perhaps you can play games with time-synchronous networking, but as far as I know t…

I'm curious if you know what, at a switch level, would actually cause drops and buffering for a 1:N (near-) saturated multicast flow. If all the packets are coming from the same source machine at (perhaps) 9.9Gbps and flowing into the switch, I would expect the switch to robustly redirect all that data with near-zero latency or packet drops to all its output ports. I don't think 10G Ethernet has "backpressure" in a w…

The nature of this kind of traffic is that it's pretty bursty. Think 100x-200x the normal packet rate in the span of a millisecond. Perfect opportunity for drops. Ultra low latency switches have tiny buffers.

Re: Thoughts on low latency trading if exchanges went full cloud

#140

Post by bitcharmer » Tue, Apr 16, 2024, 6:46 PM UTC

lucianbr wrote: post #71

posnet wrote: post #54

The biggest current limitation with cloud providers when it comes to exchange tech is the lack of real multicast support. It is rare outside of exchanges, but extremely low latency L1 multicast market data has become the backbone of exchanges, both for fairness and for scalability. Knowing you can saturate your entire network with 10G traffic and every participant will get the same market data packets at the same tim…

I find it sad that equal access between the entities doing HFT and regular Joes is not required for fairness, but god forbid one HFT having some milisecond advantage over another. That would be unfair. Can't have that.

It seems you're confused about how competition works among hft shops. There is no regulated, same latency for us. We compete for faster access just like everyone else.