LMAX Disruptor – High Performance Inter-Thread Messaging Library • Hacker News Board

Re: LMAX Disruptor – High Performance Inter-Thread Messaging Library

#41

Post by imtringued » Sat, Nov 18, 2023, 10:13 AM UTC

jiggawatts wrote: post #27

onetimeuse92304 wrote: post #15

Earlier quoted context omitted.

I built a PoC of a 5us trading system (guaranteed 5us response in every situation) for a brokerage house a long time ago, around the time of LMAX Disruptor. It was one man job and I had to start with nothing (they had no knowledge at all). Fun project and I learned a lot. * full kernel bypass (I even implemented driver for the networking hardware) * everything that could disrupt the application disabled (like SME int…

> the main insight was that rather than wait for market signals to then decide what to do, you can precalculate your responses up to and including the actual message to be sent to the exchange. I saw a talk about this dialed up to eleven: the entire processing occurred in a "smart NIC" instead of the CPU. The response would start getting sent even as the inbound packet was still being received. The go/no-go decision…

You could in principle accelerate the most CPU intensive parts of Webservers with smart NICs. gzip, TLS, JSON serdes, html templates. There are also accelerators for databases, leaving just the business logic to be executed on the CPU.

Re: LMAX Disruptor – High Performance Inter-Thread Messaging Library

#42

Post by onetimeuse92304 » Sat, Nov 18, 2023, 10:17 AM UTC

xoranth wrote: post #36

onetimeuse92304 wrote: post #28

Earlier quoted context omitted.

Thread per request can never have better load balancing between cores than a well designed, custom solution. You are essentially asking the operating system to do the scheduling for you. But the OS will never be able to do it perfectly as it has no knowledge of what your application is doing. The main advantage of OS scheduling is that you get pretty good results without having to think about it at all. Pretty good,…

> You are essentially asking the operating system to do the scheduling for you. But the OS will never be able to do it perfectly as it has no knowledge of what your application is doing. From LMAX presentations, it looks like they want you to split your application into tasks [1], define a graph of task dependencies, have each core process a particular kind of task and have task processors communicate their producers…

I know all about LMAX architecture, at least all that has been published (see my other comments for this submission).

Static allocation is a special case of scheduling. You decide which parts of the process run on which core -- the scheduling in this case is done at design or configuration time.

> On the other hand, if you have a thread per request, and allow them to migrate between cores, idle cores can steal tasks from busy ones. So in theory you could get better utilization,

Migrating your tasks between cores is nothing you can't design into your application. For example, in a typical event driven architecture where you have worker threads running each on separate core, there would be something to decide where the task is queued and usually the logic will take into account how busy particular worker thread is. The operating system does nothing in this case, what it sees is a number of threads each running on its separate core that don't need to be preempted (hopefully).

Re: LMAX Disruptor – High Performance Inter-Thread Messaging Library

#43

Post by onetimeuse92304 » Sat, Nov 18, 2023, 10:21 AM UTC

bitcharmer wrote: post #38

onetimeuse92304 wrote: post #15

Earlier quoted context omitted.

I built a PoC of a 5us trading system (guaranteed 5us response in every situation) for a brokerage house a long time ago, around the time of LMAX Disruptor. It was one man job and I had to start with nothing (they had no knowledge at all). Fun project and I learned a lot. * full kernel bypass (I even implemented driver for the networking hardware) * everything that could disrupt the application disabled (like SME int…

Appreciate you describing this as a PoC because in reality it's impossible to do In fact, you can't guarantee 5us for anything, at least not on common operating systems. You would have to run your code with the interrupt flag cleared to prevent any IPIs or hrticks getting in the way. But that would be opening a scary can of worms.

> Appreciate you describing this as a PoC because in reality it's impossible to do Absolutely not true. I explained, risk is calculated concurrently and compiled to decision tree and then inserted in the path in the form of "if X happens do Y".

The fact it was PoC has nothing to do with it, in fact it showed that yes, it is possible to do so. That's entire point of the PoC.

Re: LMAX Disruptor – High Performance Inter-Thread Messaging Library

#44

Post by onetimeuse92304 » Sat, Nov 18, 2023, 10:22 AM UTC

dajt wrote: post #39

onetimeuse92304 wrote: post #33

Earlier quoted context omitted.

Machine code isn't some kind of magical idea. It is just bytes and you can craft the bytes any way you want. In my case I wrote my own compiler that generated machine code bytes based on my needs. The compiler wasn't at all complicated. It was not general compiler that would have to be able to deal with everything you could throw at it. It only supported one model of CPU. It did not require any optimisation mechanism…

How did you inject the machine code into a running program on anything near a modern architecture? Unless you had your own OS, wouldn't code segments be RO and data segments not executable? I don't know enough about any processor created in the last 30 years to know if running bare metal without a commercial OS would allow you to not have those constraints.

If you can modify your operating system you can do pretty much anything you want (Thanks, Linus!)

Re: LMAX Disruptor – High Performance Inter-Thread Messaging Library

#45

Post by reisse » Sat, Nov 18, 2023, 10:34 AM UTC

bitcharmer wrote: post #38

onetimeuse92304 wrote: post #15

Earlier quoted context omitted.

I built a PoC of a 5us trading system (guaranteed 5us response in every situation) for a brokerage house a long time ago, around the time of LMAX Disruptor. It was one man job and I had to start with nothing (they had no knowledge at all). Fun project and I learned a lot. * full kernel bypass (I even implemented driver for the networking hardware) * everything that could disrupt the application disabled (like SME int…

Appreciate you describing this as a PoC because in reality it's impossible to do In fact, you can't guarantee 5us for anything, at least not on common operating systems. You would have to run your code with the interrupt flag cleared to prevent any IPIs or hrticks getting in the way. But that would be opening a scary can of worms.

It is absolutely possible, I've seen multiple such systems.

> In fact, you can't guarantee 5us for anything, at least not on common operating systems.

Oh yes you can, with a screen-sized kernel cmdline, some proper configuration, both hardware and software, and a bit of luck.

Re: LMAX Disruptor – High Performance Inter-Thread Messaging Library

#46

Post by belter » Sat, Nov 18, 2023, 10:36 AM UTC

pclmulqdq wrote: post #23

Every time a new generation plays with the LMAX disruptor, it's time to remind them that the modes with multiple producers/consumers can have really bad tail latency if your application's threading is not designed in the intended way. Disruptor and most other data structures that come from trading are designed to run with thread-per-core systems. This means systems where there will be no preemption during a critical…

And also time to remind that generation raised on other meandering, unfruitful roads...That is coded in the industrial strength of Java.

Re: LMAX Disruptor – High Performance Inter-Thread Messaging Library

#47

Post by reisse » Sat, Nov 18, 2023, 10:41 AM UTC

jiggawatts wrote: post #27

onetimeuse92304 wrote: post #15

Earlier quoted context omitted.

I built a PoC of a 5us trading system (guaranteed 5us response in every situation) for a brokerage house a long time ago, around the time of LMAX Disruptor. It was one man job and I had to start with nothing (they had no knowledge at all). Fun project and I learned a lot. * full kernel bypass (I even implemented driver for the networking hardware) * everything that could disrupt the application disabled (like SME int…

> the main insight was that rather than wait for market signals to then decide what to do, you can precalculate your responses up to and including the actual message to be sent to the exchange. I saw a talk about this dialed up to eleven: the entire processing occurred in a "smart NIC" instead of the CPU. The response would start getting sent even as the inbound packet was still being received. The go/no-go decision…

> The go/no-go decision was effectively just sending the final CRC bytes correctly or deliberately incorrectly, thus invalidating the outbound packet that was already 99% sent.

This trick will get you banned on some of exchanges now :)

Another one, which is public knowledge for years, and also often penalized, is to send TCP fragment with header of the message well in advance, "booking" place in the queue. Then send the finishing fragment with the real order after doing all the calculations.

Re: LMAX Disruptor – High Performance Inter-Thread Messaging Library

#48

Post by FridgeSeal » Sat, Nov 18, 2023, 11:04 AM UTC

bitcharmer wrote: post #38

onetimeuse92304 wrote: post #15

Earlier quoted context omitted.

I built a PoC of a 5us trading system (guaranteed 5us response in every situation) for a brokerage house a long time ago, around the time of LMAX Disruptor. It was one man job and I had to start with nothing (they had no knowledge at all). Fun project and I learned a lot. * full kernel bypass (I even implemented driver for the networking hardware) * everything that could disrupt the application disabled (like SME int…

Appreciate you describing this as a PoC because in reality it's impossible to do In fact, you can't guarantee 5us for anything, at least not on common operating systems. You would have to run your code with the interrupt flag cleared to prevent any IPIs or hrticks getting in the way. But that would be opening a scary can of worms.

> you can't guarantee 5us for anything, at least not on common operating systems

I don’t doubt it, but from my reading of OP’s post, it sounds they were skipping most of the kernel and OS.

Re: LMAX Disruptor – High Performance Inter-Thread Messaging Library

#49

Post by marginalia_nu » Sat, Nov 18, 2023, 11:17 AM UTC

nine_k wrote: post #35

citrin_ru wrote: post #34

Earlier quoted context omitted.

Easy to implement.

But does anyone run an OS thread per request unironically? I thought that nearly every request-response server implementation would use a thread pool. The best, like Erlang, can give you the feeling of arbitrarily many extremely cheap threads, while also running on a thread pool.

As a devil's advocate argument, if you're doing serverside rendering, and basically getting 1 request per visit, sure there's overhead, but even like a landslide HN death hug is only a handful requests per second. A raspberry pi could feasibly serve that traffic spawning 1 thread per request.

... not that I think anyone is doing this outside of maybe some hobbyist building their own HTTP server for fun.

Re: LMAX Disruptor – High Performance Inter-Thread Messaging Library

#50

Post by marginalia_nu » Sat, Nov 18, 2023, 11:24 AM UTC

vivzkestrel wrote: post #18

onetimeuse92304 wrote: post #15

Earlier quoted context omitted.

I built a PoC of a 5us trading system (guaranteed 5us response in every situation) for a brokerage house a long time ago, around the time of LMAX Disruptor. It was one man job and I had to start with nothing (they had no knowledge at all). Fun project and I learned a lot. * full kernel bypass (I even implemented driver for the networking hardware) * everything that could disrupt the application disabled (like SME int…

what language do you think would be the backbone for such a system? C/C++/Golang or something high level like node.js/Java

I wouldn't bundle C with Golang and Java with Node. Golang and Java are roughly a tier on their own. Java is definitely a thing in this space[1], although it's a fairly unidiomatic style of Java that reduces allocations and puts a lot of emphasis on consistent low latency.

[1] e.g. https://marketswiki.com/wiki/TRADExpress ; though they've been absorbed into Nasdaq now and my visibility into CFT ended with that, no idea how much of their software is still running