Why on earth would exchanges go full cloud when running the trading infrastructure reliably and predictably is their whole reason for existing?
Thoughts on low latency trading if exchanges went full cloud
81–90 of 184 posts
Re: Thoughts on low latency trading if exchanges went full cloud
#82Earlier quoted context omitted.
How do you tie break? If there are more sellers than buyers (or vice versa) at the clearing price?
The same way you would without a clock I guess? You could match what you can distributed equally and leave the rest unsettled. You could let people decide whether to roll-over the partial bid into a new bid on the next clock or to cancel unsettled. You could clock to something both very fast on a human scale (50ms), quick enough it'd still feel instant but slow enough that it could reduce HFT silliness and need for e…
> You could match what you can distributed equally and leave the rest unsettled.
Equally per market participant? Do large participant like banks trade same amount as retail investor one trade at a time? Per quantity? HFT will time the end of the interval and decide to place a large order or not.
Re: Thoughts on low latency trading if exchanges went full cloud
#83The biggest current limitation with cloud providers when it comes to exchange tech is the lack of real multicast support. It is rare outside of exchanges, but extremely low latency L1 multicast market data has become the backbone of exchanges, both for fairness and for scalability. Knowing you can saturate your entire network with 10G traffic and every participant will get the same market data packets at the same tim…
I find it sad that equal access between the entities doing HFT and regular Joes is not required for fairness, but god forbid one HFT having some milisecond advantage over another. That would be unfair. Can't have that.
Because average Joes don't do algorithmic trading, and if they do it not at the level that HFT does. Not even all the big financial players care about HFT and millisecond timing, so they're in the same boat.
Re: Thoughts on low latency trading if exchanges went full cloud
#84Earlier quoted context omitted.
> All this HFT feels wasteful and bad for 'regular' human investors. Quite the opposite, thanks to the tough competition the market makers are setting the bid/asks spreads as minimal as possible. Which leads to less costs for human investors, pension funds, insurance companies etc. I used to be a market maker in the 90's before HFT took off. The margins we kept sometimes felt like a rip off but customers had no other…
>Quite the opposite, thanks to the tough competition the market makers are setting the bid/asks spreads as minimal as possible. Which leads to less costs for human investors, pension funds, insurance companies etc. It's not automatically the case that the disappeared margins & thinning of bid/asks have been shared equitably between the trading firms and customers. Take two exaggerated markets for example: 1) No HFTs:…
This is entirely false and ignores Reg NMS. Everyone must execute at the NBBO. As well customer orders are often given better and tighter prices than other market participants. HFT firms will often offer them better than NBBO prices. As well, none of these prices you quote would not exist without market makers, its just now the fact that to be a market maker you must be an HFT firm as well due to the scale that is now required.
Re: Thoughts on low latency trading if exchanges went full cloud
#85The biggest current limitation with cloud providers when it comes to exchange tech is the lack of real multicast support. It is rare outside of exchanges, but extremely low latency L1 multicast market data has become the backbone of exchanges, both for fairness and for scalability. Knowing you can saturate your entire network with 10G traffic and every participant will get the same market data packets at the same tim…
> Knowing you can saturate your entire network with 10G traffic and every participant will get the same market data packets at the same time[0]
Hold on a second. Multicast is nifty, but it does not perform miracles. If you operate a 10G multicast network and actually saturate it, you will experience drops and buffering-induced delays. Perhaps you can play games with time-synchronous networking, but as far as I know the exchanges don’t do this, and it likely needs special hardware.
The point of 10G multicast is to use simple, standard (but complex to configure!) equipment to distribute much less than 10Gbps simultaneously.
Re: Thoughts on low latency trading if exchanges went full cloud
#86Earlier quoted context omitted.
> lack of real multicast support Yup, this is a problem for us in GCP today even outside of trading. I don't know how Pub/Sub works for them.
Pub/sub systems in unicast-only environments are very complex distributed systems to handle the load involved in fan-out routing while maintaining a global order. I had an interviewer once get annoyed with me for suggesting using multicast to solve the fan-out part of a pub/sub system, which made the global ordering part small and simple. We lost a lot by thinking of HTTP as the one true level of network abstraction.
A reliable multicast network that preserves global order even during maintenance and doesn’t drop packets is not something you will find off the shelf.
A reliable multi-tenant multicast network also appears to be a rare beast. I’ve only heard of it in finance, and that’s only because it’s private and expensive and all the participants need to be generally nice to each other because it’s a repeated game and the operator can literally pull the plug if the rules are broken.
Re: Thoughts on low latency trading if exchanges went full cloud
#87If a big exchange goes to the cloud it won’t look like a regular company setting up an aws account and getting a bunch of ec2 instances in us-east-1. They would at least have dedicated racks. I suspect the provider would end up with a plan where traders can get servers that all have the same network distance from the exchange’s nics (down to the same length of fiber).
I think and exchange would use AWS Outposts to get hardware in their data center.
Re: Thoughts on low latency trading if exchanges went full cloud
#88Earlier quoted context omitted.
It doesn't have to be arbitrary machine code. eBPF / WASM coupled with a standard library supplied by the exchange. (Plus the exchange can run it in a VM.) > Also if two people want to make the same trade, who gets it? Whoever pays more currently, right? So it can be uniform random and folks can pay for better than random chance, etc.
Or we could just.. not do that? It provides no benefit and a host of downsides
It seems a lot more elegant/efficient/sane to me than trying to squeeze more and more racks into one building. So that's why I'm asking, as I think the benefits are clear, much better scalability, fairness (or getting as close to it as the exchange wants), cheaper (no need for fancy hardware), probably it would attract more market participants (lower barriers to entry).
It's the same hypothetical "EC2 model" without the meta-game of trying to get closer to the cores the exchange runs at a given time.
Can you elaborate on the downsides besides security?
Re: Thoughts on low latency trading if exchanges went full cloud
#89At first glance, it seems this would even the playing field. However, large players will allocate resources to spinning up instances and overloading machines. Similar to how we're seeing the DDOS shenanigans going on in crypto.
Net-net, it still benefits startup quant shops and sophisticated independents. Most retail isn't doing HFT or really any quant. But for people wanting to have their own shops, this is a better version than having to build hardware and colo.
Re: Thoughts on low latency trading if exchanges went full cloud
#90Earlier quoted context omitted.
> All this HFT feels wasteful and bad for 'regular' human investors. Quite the opposite, thanks to the tough competition the market makers are setting the bid/asks spreads as minimal as possible. Which leads to less costs for human investors, pension funds, insurance companies etc. I used to be a market maker in the 90's before HFT took off. The margins we kept sometimes felt like a rip off but customers had no other…
>Quite the opposite, thanks to the tough competition the market makers are setting the bid/asks spreads as minimal as possible. Which leads to less costs for human investors, pension funds, insurance companies etc. It's not automatically the case that the disappeared margins & thinning of bid/asks have been shared equitably between the trading firms and customers. Take two exaggerated markets for example: 1) No HFTs:…
In the United States at least both scenarios you mentioned are illegal. Market makers are not just sitting in the middle of orders. They buy without a seller lined up and then fill orders from their own inventory (or route orders to an exchange in the case where they can't fill a buy order from their own inventory). In cases where they route to an exchange they are required by law to fill the order at the lowest price available. Typically they fill orders at better prices than what you can get on an exchange. So you, as a retail investor, are actually getting better prices than you would if your broker just filled orders on an exchange.
How the price improvement gets allocated is complicated. Some of the price improvement goes to the broker (in the form a payment-for-order-flow) and some goes to the actual investor (you). But in either case the retail investors are strictly better off.