Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LMAX Disruptor – High Performance Inter-Thread Messaging Library

lmax-exchange.github.io

Post reply on HN
Print view View original

61–70 of 89 posts

Re: LMAX Disruptor – High Performance Inter-Thread Messaging Library

#61

Post by sgift »

gmfawcett wrote: post #56
nine_k wrote: post #35

Earlier quoted context omitted.

But does anyone run an OS thread per request unironically? I thought that nearly every request-response server implementation would use a thread pool. The best, like Erlang, can give you the feeling of arbitrarily many extremely cheap threads, while also running on a thread pool.

> does anyone run an OS thread per request unironically? Of course they do. There are loads of appropriate applications. Heck, people still run CGI programs unironically.

And if your use case allows it it is a great model. Easy to setup, easy to debug, easy to run.

Also, why I love the new virtual threads in Java. Will they work as good as hoped? No idea. Probably not for all use cases. But the direction to say: You know what, threads are great to program and debug compared to the alternatives, maybe let's find a way to make their performance better instead of putting up with async , is so refreshing.

Re: LMAX Disruptor – High Performance Inter-Thread Messaging Library

#62

Post by bob1029 »

I love this pattern. There are many problems that fit it quite well once you start thinking in these terms - Intentionally delaying execution over (brief amounts of) time in order to create batching opportunities which leverage the physical hardware's unique quirks.

Any domain with synchronous/serializable semantics can modeled as a single writer, with an MPSC queue in front of it. Things like game worlds, business decision systems, database engines, etc. fit the mold fairly well.

The busy-spin strategy can be viewed as a downside, but I can't ignore the latency advantages. In my experiments where I have some live "analog" input like a mouse, the busy wait strat feels 100% transparent. I've tested it for hours without it breaking into the millisecond range (on windows 10!). For gaming/real-time UI cases, you either want this or yield. Sleep strategies are fine if you can tolerate jitter in the millisecond range.

Re: LMAX Disruptor – High Performance Inter-Thread Messaging Library

#63

Post by mgaunard »

I built trading systems for LMAX exchanges. Their technology seems quite far from the state of the art to me.

I didn't know they even claimed to attempt being the fastest exchange in the world. They're very far from being so and it's quite clear that there are architectural decisions in that platform that would prevent that.

Re: LMAX Disruptor – High Performance Inter-Thread Messaging Library

#64

Post by quadrature »

mgaunard wrote: post #63

I built trading systems for LMAX exchanges. Their technology seems quite far from the state of the art to me. I didn't know they even claimed to attempt being the fastest exchange in the world. They're very far from being so and it's quite clear that there are architectural decisions in that platform that would prevent that.

What kinds of issues did you see?. Do you think there are better alternatives to the disruptor ?

Re: LMAX Disruptor – High Performance Inter-Thread Messaging Library

#65

Post by onetimeuse92304 »

twic wrote: post #53
defrost wrote: post #19

Earlier quoted context omitted.

Each to their own but if you read and understand the comment above they're describing a dedicated OS for the task .. so think about what you'd choose to write a small task dedicated OS with. Simple C is most likely, ASM is possible, a language such as OCaml generating C to hook into the low level buffers would be intriguing ... the list is long and largely determined by the experience preference of whoever tackles it…

My team did it in Rust. Others in our company have done it in C++, making heavy use of available wizardry. It's widespread in the industry to do it in Java, albeit a strange subset of Java where you avoid most of the features. I'd be amazed if anyone did it entirely in C. The productivity is just too low.

The question really is "why?" Most systems have relatively few hotspots or can be reorganised to have relatively few hotspots. You don't need to write everything in a low level language to just have those few things run fast, unless you are also facing other constraints like low memory.

I have better uses of my time than write megabytes of pointer manipulation in C. That's why I prefer high level languages that can easily coexist with C for those few parts that really benefit.

Re: LMAX Disruptor – High Performance Inter-Thread Messaging Library

#66

Post by onetimeuse92304 »

FridgeSeal wrote: post #48
bitcharmer wrote: post #38

Earlier quoted context omitted.

Appreciate you describing this as a PoC because in reality it's impossible to do In fact, you can't guarantee 5us for anything, at least not on common operating systems. You would have to run your code with the interrupt flag cleared to prevent any IPIs or hrticks getting in the way. But that would be opening a scary can of worms.

> you can't guarantee 5us for anything, at least not on common operating systems I don’t doubt it, but from my reading of OP’s post, it sounds they were skipping most of the kernel and OS.

Yes. You really don't want your real time threads to have anything to do with the OS after startup sequence. Significant part of the project was learning about various ways a piece of code can be preempted.

Once you get it done, it really is all really nice and predictable. And also a fire hazard because you have disabled things like SMI interrupts that are used by the CPU to figure out if it is overheating...

Re: LMAX Disruptor – High Performance Inter-Thread Messaging Library

#67

Post by yencabulator »

nine_k wrote: post #35
citrin_ru wrote: post #34

Earlier quoted context omitted.

Easy to implement.

But does anyone run an OS thread per request unironically? I thought that nearly every request-response server implementation would use a thread pool. The best, like Erlang, can give you the feeling of arbitrarily many extremely cheap threads, while also running on a thread pool.

As far as comparisons to thread-per-core go, thread per request applies whether it's an OS thread or a green thread or a Rust async function compiled into a state machine. Anything that multiplexes per-request contexts into a lesser amount of cores(/OS threads) has the same trade-offs, the difference is more on the easy-vs-optimized spectrum. Thread-per-core with fixed workloads behaves differently than all of those.

Here's an example difference: in thread-per-request, any global state can be accessed from "anywhere", and thus you end up with locks, reference counts, GC, and what not. In thread-per-core, global state is sharded across cores and never accessed "from the outside", and thus needs no locks/atomics (beyond the messaging primitive).

Re: LMAX Disruptor – High Performance Inter-Thread Messaging Library

#68

Post by reisse »

SkyArrow wrote: post #58
reisse wrote: post #47

Earlier quoted context omitted.

> The go/no-go decision was effectively just sending the final CRC bytes correctly or deliberately incorrectly, thus invalidating the outbound packet that was already 99% sent. This trick will get you banned on some of exchanges now :) Another one, which is public knowledge for years, and also often penalized, is to send TCP fragment with header of the message well in advance, "booking" place in the queue. Then send…

This bears the question of how does an exchange efficiently detect, log and take action against these kinds of behaviours without increasing its own latency too much and (perhaps?) affecting the market? Does it even matter if a centralised exchange increases its own latency when all market participants have to go through it? I can only think of the case when a security is listed on multiple exchanges, where the laten…

Exchanges rarely care about their absolute latency. The latency race is for the best place in the order entry queue. As soon as the order is queued for sequential processing by the risk checker or the matching engine, the race is over. I've seen places where you needed sub-microsecond tick-to-order latency to win the race for the front of the queue, but the actual risk check and matching took tens of milliseconds.

They do care about throughput and providing fair conditions to all of the participants, though. On busiest derivatives exchanges this means resorting to FPGAs for initial checks.

Then, every message send to the exchange is clearly traceable. In some cases participants have dedicated physical lines. When the exchange sees increased rate of malformed packets from a single line or from a certain participant, they just cut it off and call the contact person from the participant (trader/broker) side to demand explanation.

Re: LMAX Disruptor – High Performance Inter-Thread Messaging Library

#69

Post by eatonphil »

mgaunard wrote: post #63

I built trading systems for LMAX exchanges. Their technology seems quite far from the state of the art to me. I didn't know they even claimed to attempt being the fastest exchange in the world. They're very far from being so and it's quite clear that there are architectural decisions in that platform that would prevent that.

Where should one look/read about for state of the art?

Re: LMAX Disruptor – High Performance Inter-Thread Messaging Library

#70

Post by mgaunard »

quadrature wrote: post #64
mgaunard wrote: post #63

I built trading systems for LMAX exchanges. Their technology seems quite far from the state of the art to me. I didn't know they even claimed to attempt being the fastest exchange in the world. They're very far from being so and it's quite clear that there are architectural decisions in that platform that would prevent that.

What kinds of issues did you see?. Do you think there are better alternatives to the disruptor ?

I don't particularly know anything about this disruptor, and it being in Java kinda biases me towards dismissing it out of hand (no one does serious systems programming in Java).

From a quick reading it's just a standard spmc queue with some mpmc capabilities. Queues (preferably lock-free and bounded) are a basic component of any low-latency distributed software system. They seem to understand the basics right, nothing too outstanding, some decisions quite suboptimal.

Myself I use spsc task queues for inter-thread communication (because in that scenario you know who you're sending tasks to, and you can easily just attach multiple spsc queues for pseudo-mpsc capabilities), and mpmc message queues for inter-process communication (because that scenario is more of a message bus, and you don't know who's talking to who).

I have built these kinds of things many times together with bespoke threading and scheduling models, as have others at all of the trading shops I've seen, so I'd say it's a pretty standard thing in the industry.

Open-source frameworks of interest would be Seastar or DPDK.

However, while a good threading model helps, it's far from sufficient to be the highest performance trading exchange. You also need to think hard about networking, be it the protocols, the software, the hardware and the topology. For example key factors in trading are deterministically publishing data to all participants at the same time, ensuring private information is not published before its public equivalent, making sure that whoever sent their packet first gets processed first. Even something as simple as the kind of switches you use has a huge impact.

Post reply on HN