How exchanges turn order books into distributed logs

These distributed sequencer solutions are for resilience, and they add a lot of latency because each node needs to do something like RAFT. Exchanges generally don't care aggressively about low latency, they care about resilience and fairness. It's the hedge funds etc looking for an edge.

One thing often missed here is that most orders, even from most hedge funds and prop trading shops, still go via broker systems. Direct Market Access is getting more common but it's often a pain in the arse from a regulatory and disclosures perspective, and means you lose out on short locate (shares that you can borrow from your broker to short sell).

"Sponsored Access", where you connect directly to an exchange but your broker monitors your activity via a drop copy, is a happy middle ground.

Surprisingly though, I've heard of at least one trading venue where going direct is slower, because the venues own risk checks are slower than the the ones implemented by at least one broker, and the broker themselves are allowed to bypass the risk checks put in place at the exchange for general DMA clients. "Direct" is clearly subject to negotiation.

I've also heard of brokers who tried to implement their gateways in FPGA, and have later shuttered the project, having gone back to relatively slow software gateways for the flexibility.

A lot of trading still happens via FIX, which is a slow ASCII protocol. Most prop shops will have aggressively optimized FIX parsers and serialisers out of necessity.

People think all trading happens in these elite, bleeding-edge hardcore sub-microsecond systems, but a lot of it is just dogshit.

Things are a bit more optimised in the derivatives space because of the insane volumes (Options trading just for US equities is easily into the petabytes of storage per year).