This is interesting but also just hilarious at a meta level. I was a “low frequency” ie manual fundamental based hedge fund investor for many years. In general I think hft is a net benefit to liquidity when done in compliance with the text and spirit of regulations. But no real world allocation of resources is improved by having to game transactions to this level of time granularity. This is just society pouring resources down a zero sum black hole. Open to hearing contrary views of course.
How exchanges turn order books into distributed logs
31–40 of 73 posts
Re: How exchanges turn order books into distributed logs
#32This is interesting but also just hilarious at a meta level. I was a “low frequency” ie manual fundamental based hedge fund investor for many years. In general I think hft is a net benefit to liquidity when done in compliance with the text and spirit of regulations. But no real world allocation of resources is improved by having to game transactions to this level of time granularity. This is just society pouring reso…
I've been wondering if the stock market would be more efficient if trades executed only every instead of continuously, i.e. every 1 second an opening trade style cross book clearance happens. Orders would have to be on the book for a full interval to execute to prevent last millisecond rushes at the end of an interval
I'm probably missing some second order effects but it feels like this would mitigate the need for race to the bottom latencies and would also provide protection against fat fingered executions in that every trading algorithm would have a full second to arbitrage it
Re: How exchanges turn order books into distributed logs
#33Smells of AI writing: "Timestamps aren't enough. Exchanges need a stronger ordering primitive." etc
Interesting comment, I "felt" the ai too in an undescribable way. What are some obvious tells ?
The incessant bullet lists and the conclusion titled "Conclusion" give it away. And above all, the complete lack of "voice". You can tell when a human is speaking and when a sanitised amorphous blob of averageness is speaking.
Re: How exchanges turn order books into distributed logs
#34This article both undersells and oversells the technical challenge exchanges solve. First, it is of course possible to apply horizontal scaling through sharding. My order on Tesla doesn't affect your order on Apple, so it's possible to run each product on its own matching engine, its own set of gateways, etc. Most exchanges don't go this far: they might have one cluster for stocks starting A-E, etc. So they don't eve…
> My order on Tesla doesn't affect your order on Apple not necessarily many exchanges allow orders into one instrument to match on another (very, very common on derivatives exchanges)
[deleted]
Re: How exchanges turn order books into distributed logs
#35This is interesting but also just hilarious at a meta level. I was a “low frequency” ie manual fundamental based hedge fund investor for many years. In general I think hft is a net benefit to liquidity when done in compliance with the text and spirit of regulations. But no real world allocation of resources is improved by having to game transactions to this level of time granularity. This is just society pouring reso…
I've been wondering if the stock market would be more efficient if trades executed only every instead of continuously, i.e. every 1 second an opening trade style cross book clearance happens. Orders would have to be on the book for a full interval to execute to prevent last millisecond rushes at the end of an interval I'm probably missing some second order effects but it feels like this would mitigate the need for ra…
You could do this but the cost would be wider bid/ask spreads for all market participants. If you make it harder for market makers to hedge their position, they will collect a larger spread to account for that. A whole lot of liquidity can disappear in a second when news hits.
I’d rather have penny-wide spreads on SPY than restrict trading speed for HFTs. Providing liquidity is beneficial to everyone, even if insane amounts of money are spent by HFTs to gain an edge.
Re: How exchanges turn order books into distributed logs
#36Smells of AI writing: "Timestamps aren't enough. Exchanges need a stronger ordering primitive." etc
yes, this is AI-assisted writing, I'm not hiding from it. but this isn't just copy-pasting. I still spend up to 15 hours working on each article.
Re: How exchanges turn order books into distributed logs
#37Earlier quoted context omitted.
why would market data go through the sequenced stream on an exchange? for an exchange: market data is a projection of the order book, an observer that sits on the stream but doesn't contribute to it and client ports have rate limits
B/c, by design, you want the archived stream of events to include everything. e.g. a lot of these systems have a "replay" node that can be used by components that just restarted. You want the replay to include ALL of the messages seen so you can rebuild the state at any given point. (There are, of course, tradeoffs to this so I'm just commenting on the "single sequencer" design philosophy)
by definition: an exchange doesn't need any reference to outside market data
even for systems built on a sequencer which do (e.g. an OMS), the volume is too large
the usual strategy is for processes which require it, is to sample it and them stamp it on commands
which maintains the invariants
(my background: I have been a developer on one of Mike Blum's original sequencers)
Re: How exchanges turn order books into distributed logs
#38This is interesting but also just hilarious at a meta level. I was a “low frequency” ie manual fundamental based hedge fund investor for many years. In general I think hft is a net benefit to liquidity when done in compliance with the text and spirit of regulations. But no real world allocation of resources is improved by having to game transactions to this level of time granularity. This is just society pouring reso…
I've been wondering if the stock market would be more efficient if trades executed only every instead of continuously, i.e. every 1 second an opening trade style cross book clearance happens. Orders would have to be on the book for a full interval to execute to prevent last millisecond rushes at the end of an interval I'm probably missing some second order effects but it feels like this would mitigate the need for ra…
this is exactly what many dark pools do
"continuous periodic auctions"
Re: How exchanges turn order books into distributed logs
#39> Pipelined replication: the sequencer assigns a sequence number immediately and ships the event to replicas in parallel. Matching doesn't wait for the replicas to acknowledge.
How is this avoiding data loss if the lead sequencer goes down after acking but without the replica receiving the write?
Re: How exchanges turn order books into distributed logs
#40Earlier quoted context omitted.
What kind of founding ethos doesn't allow tracking internal latency? Is their founding ethos "Never Admit Responsibility?"; "Never Leave A Paper Trail?" This company's official ethical foundation is "Don't Get Caught."
From the wiki about IEX: "It was founded in 2012 in order to mitigate the effects of high-frequency trading." I can see how they don't want to track internal latency as part of that, or at least not share those numbers with outsiders. That just encourages high frequency traders again.
One would hope for a more technical solution to HFT than willful ignorance lol. For example, they could batch up orders every second and randomize them.