How exchanges turn order books into distributed logs • Hacker News Board

Re: How exchanges turn order books into distributed logs

#51

Post by atomicnumber3 » Sat, Dec 13, 2025, 5:55 PM UTC

pants2 wrote: post #40

WJW wrote: post #22

Earlier quoted context omitted.

From the wiki about IEX: "It was founded in 2012 in order to mitigate the effects of high-frequency trading." I can see how they don't want to track internal latency as part of that, or at least not share those numbers with outsiders. That just encourages high frequency traders again.

One would hope for a more technical solution to HFT than willful ignorance lol. For example, they could batch up orders every second and randomize them.

I worked in HFT. (Though am now completely out of fintech and have no skin in the game). "Flash Boys" traditional HFT is dead already, the trade collapsed in 2016-2018 when both larger institutions got less dumb with order execution, and also several HFTs "switched sides" and basically offered "non-dumb order execution" as a service to any institutions who were unable to play the speed game themselves. Look at how Virtu's revenue changed from mostly trading to mostly order execution services over that time period.

Flash Boys was always poorly researched and largely ignorant of actual market microstructure and who the relevant market participants were, but it also aged quite poorly as all of their "activism" was useless because the market participants just all smartened up purely profit-driven.

If you want to be activist about something, the best bet for 2026 is probably that so much volume is moving off the lit exchanges into internal matching and it degrades the quality of price discovery happening. But honestly even that's a hard sell because much of that flow is "dumb money" just wanting to transact at the NBBO.

Actually, here's the best thing to be upset about: apps gamifying stock trading / investing into basically SEC-regulated gambling.

Re: How exchanges turn order books into distributed logs

#52

Post by SOTGO » Sat, Dec 13, 2025, 6:03 PM UTC

thijson wrote: post #50

The article says it's not enough to accurately timestamp orders at the various order entry portals. I didn't understand why that's not enough. GPS can provide fairly accurate timestamps. There's a few other GLONASS systems as well for extra reliability.

It's probably possible to use timestamps, but I suppose you would have to handle ties in more places, with sequence numbers you only break ties once. It appears that the FIX specifications allows up to microsecond precision, but given the volume of messages it's still likely a problem. It's also easier to work with integer sequence numbers than timestamps, but that's also a small consideration.

Re: How exchanges turn order books into distributed logs

#53

Post by extraduder_ire » Sat, Dec 13, 2025, 6:57 PM UTC

thijson wrote: post #50

The article says it's not enough to accurately timestamp orders at the various order entry portals. I didn't understand why that's not enough. GPS can provide fairly accurate timestamps. There's a few other GLONASS systems as well for extra reliability.

GNSS is the generic term, GLONASS is the name for the Russian system.

Re: How exchanges turn order books into distributed logs

#54

Post by Nextgrid » Sat, Dec 13, 2025, 7:20 PM UTC

bwfan123 wrote: post #25

croemer wrote: post #11

Smells of AI writing: "Timestamps aren't enough. Exchanges need a stronger ordering primitive." etc

Interesting comment, I "felt" the ai too in an undescribable way. What are some obvious tells ?

"It isn't , it's " and variations thereof are the biggest tell.

Re: How exchanges turn order books into distributed logs

#55

Post by thadt » Sat, Dec 13, 2025, 7:21 PM UTC

dboreham wrote: post #24

alexpotato wrote: post #4

At a past job (hedge fund), my role was to co-ordinate investigations into why latency may have changed when sending orders. A couple of quants had built a random forest regression model that could take inputs like time of day, exchange, order volume etc and spit out an interval of what latency had historically been in that range. If the latency moved outside that range, an alert would fire and then I would co-ordina…

> they once swapped out a cable that was a few feet longer than the older cable and that's why the latency increased That was not why. Possibly the cable made a difference (had an open circuit that made the NICs back down to a lower speed; noisy leading to retransmissions) but it wasn't the length per se.

Well, it depends on the granularity of the time scale right? When you're measuring milliseconds, then the cable length probably isn't a thing factoring into your latency calculation.

When we're measuring time on the scale of nanoseconds then, yes, cable length is definitely something we care about and will reliably show up in measurements. In some situations, we not only care about the cable length, but also its temperature.

Re: How exchanges turn order books into distributed logs

#56

Post by stuxnet79 » Sat, Dec 13, 2025, 7:39 PM UTC

philipov wrote: post #20

alexpotato wrote: post #4

At a past job (hedge fund), my role was to co-ordinate investigations into why latency may have changed when sending orders. A couple of quants had built a random forest regression model that could take inputs like time of day, exchange, order volume etc and spit out an interval of what latency had historically been in that range. If the latency moved outside that range, an alert would fire and then I would co-ordina…

What kind of founding ethos doesn't allow tracking internal latency? Is their founding ethos "Never Admit Responsibility?"; "Never Leave A Paper Trail?" This company's official ethical foundation is "Don't Get Caught."

The founder Brad Katsuyama talks about his background and motivation for starting the company here - https://www.youtube.com/watch?v=N9hoqFpDjVs

It might add a bit of color to this conversation.

Re: How exchanges turn order books into distributed logs

#57

Post by lopatin » Sat, Dec 13, 2025, 7:40 PM UTC

alexpotato wrote: post #4

At a past job (hedge fund), my role was to co-ordinate investigations into why latency may have changed when sending orders. A couple of quants had built a random forest regression model that could take inputs like time of day, exchange, order volume etc and spit out an interval of what latency had historically been in that range. If the latency moved outside that range, an alert would fire and then I would co-ordina…

Can you talk a bit more about the incentives to trade latency sensitive strategies on IEX in the first place? Is it still lucrative for its liquidity despite them artificially slowing down orders? Does a meta game evolve with HFTs all working around their system, essentially making it still a HFT playground but with extra steps? Do you think their unexpected latency increase for you guys was intentional, to free the water from sharks?

Re: How exchanges turn order books into distributed logs

#58

Post by kasey_junk » Sat, Dec 13, 2025, 7:45 PM UTC

TheJoeMan wrote: post #27

8cvor6j844qw_d6 wrote: post #21

Earlier quoted context omitted.

May I know if someone with no trading knowledge can get into this field? Or do new hires that you've seen generally have some background knowledge on related to trading, etc.? I did consider applying for a role in a very similar field, but figured I'll be fighting an uphill battle with no knowledge in trading/stock market/etc.

You know how coders are expected to grind out leetcode interviews? For the finance fields, a common interview topic is what you read in “The Journal” (WSJ). So just stay on top of it for a few weeks, see some trends, etc.

I’ve worked in finance for 25 years and never even heard of this coming up in an interview.

Re: How exchanges turn order books into distributed logs

#59

Post by neonbrain » Sat, Dec 13, 2025, 7:58 PM UTC

Workaccount2 wrote: post #43

pants2 wrote: post #40

Earlier quoted context omitted.

One would hope for a more technical solution to HFT than willful ignorance lol. For example, they could batch up orders every second and randomize them.

This is what should happen, because what the game actually being played is to profit off those who cannot react fast enough to news event, rather than profit off those who mispriced their order. Or leave things in place, but put a 1 minute transaction freeze during binary events, and fill the order book during that time with no regard for when an order was placed, just random allocation of order fills coming out of t…

This isn't a good approach because it assumes there are no market makers on trading venues, and that they (as well as exchanges) do not compete for order flow. Also, maybe you haven't noticed, but stocks are often frozen during news announcements by regulatory request, so such pauses are already in place and are designed to maintain market integrity, not disrupt it with arbitrary fills.

Re: How exchanges turn order books into distributed logs

#60

Post by genidoi » Sat, Dec 13, 2025, 7:58 PM UTC

rundef wrote: post #36

croemer wrote: post #11

Smells of AI writing: "Timestamps aren't enough. Exchanges need a stronger ordering primitive." etc

yes, this is AI-assisted writing, I'm not hiding from it. but this isn't just copy-pasting. I still spend up to 15 hours working on each article.

Considering that you chose to not include your name or even a HN username in the byline of the article, there is an argument to be made that you are, in fact, hiding from it.