Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LMAX Disruptor – High Performance Inter-Thread Messaging Library

lmax-exchange.github.io

Post reply on HN
Print view View original

81–89 of 89 posts

Re: LMAX Disruptor – High Performance Inter-Thread Messaging Library

#81

Post by gmfawcett »

yencabulator wrote: post #67
nine_k wrote: post #35

Earlier quoted context omitted.

But does anyone run an OS thread per request unironically? I thought that nearly every request-response server implementation would use a thread pool. The best, like Erlang, can give you the feeling of arbitrarily many extremely cheap threads, while also running on a thread pool.

As far as comparisons to thread-per-core go, thread per request applies whether it's an OS thread or a green thread or a Rust async function compiled into a state machine. Anything that multiplexes per-request contexts into a lesser amount of cores(/OS threads) has the same trade-offs, the difference is more on the easy-vs-optimized spectrum. Thread-per-core with fixed workloads behaves differently than all of those.…

> In thread-per-core, global state is sharded across cores and never accessed "from the outside"

...or possibly it's accessed from everywhere, and locks are needed again. What you're describing is kind of an ideal application architecture, not a hard rule about thread pinning.

Re: LMAX Disruptor – High Performance Inter-Thread Messaging Library

#83

Post by gpderetta »

mgaunard wrote: post #80
gpderetta wrote: post #75

Earlier quoted context omitted.

It is a broadcast queue. Consumers all receive the same data, so they don't contend to pop elements. And producers never wait for consumers. Slow consumers have to deal with missing packets. At the end of the day, it is a specialised ring buffer that happens to be useful for many use cases.

That just sounds like a normal spmc queue as I said above. Of course they receive the same elements, or it wouldn't be multi-consumer. Of course producers don't wait for consumers, they don't even need to be aware of how many there are or where they are. But even in a system where you'd know (e.g. publishing data to a bunch a TCP connections), it would he a very bad idea to stall production -- handling back pressure…

Topically in an MC queue, a consumer "consumes" an element and won't be available for other consumers. For example a job queue. Also typically queues either grow unbounded or pushes fail.

So no, I wouldn't say that the disruptor is a normal SPMC or MPMC queue as it semantics are different.

But yes, it has pretty much the same characteristics of UDP.

Re: LMAX Disruptor – High Performance Inter-Thread Messaging Library

#84

Post by nine_k »

gmfawcett wrote: post #81
yencabulator wrote: post #67

Earlier quoted context omitted.

As far as comparisons to thread-per-core go, thread per request applies whether it's an OS thread or a green thread or a Rust async function compiled into a state machine. Anything that multiplexes per-request contexts into a lesser amount of cores(/OS threads) has the same trade-offs, the difference is more on the easy-vs-optimized spectrum. Thread-per-core with fixed workloads behaves differently than all of those.…

> In thread-per-core, global state is sharded across cores and never accessed "from the outside" ...or possibly it's accessed from everywhere, and locks are needed again. What you're describing is kind of an ideal application architecture, not a hard rule about thread pinning.

What you need to share, you may adorn with locks, queues, etc. But ideally you share very little.

"Possibly accessed from anywhere" is a bad design in general, and unacceptable in realtime processing, where you need to know access patterns exactly.

Re: LMAX Disruptor – High Performance Inter-Thread Messaging Library

#85

Post by bitcharmer »

FridgeSeal wrote: post #48
bitcharmer wrote: post #38

Earlier quoted context omitted.

Appreciate you describing this as a PoC because in reality it's impossible to do In fact, you can't guarantee 5us for anything, at least not on common operating systems. You would have to run your code with the interrupt flag cleared to prevent any IPIs or hrticks getting in the way. But that would be opening a scary can of worms.

> you can't guarantee 5us for anything, at least not on common operating systems I don’t doubt it, but from my reading of OP’s post, it sounds they were skipping most of the kernel and OS.

Every respectable HFT shop does that. What you can't do is achieve <5us execution latency at 100% percentile. You'd have to disable LAPIC interrupts for that which I doubt they did.

Re: LMAX Disruptor – High Performance Inter-Thread Messaging Library

#86

Post by mgaunard »

gpderetta wrote: post #83
mgaunard wrote: post #80

Earlier quoted context omitted.

That just sounds like a normal spmc queue as I said above. Of course they receive the same elements, or it wouldn't be multi-consumer. Of course producers don't wait for consumers, they don't even need to be aware of how many there are or where they are. But even in a system where you'd know (e.g. publishing data to a bunch a TCP connections), it would he a very bad idea to stall production -- handling back pressure…

Topically in an MC queue, a consumer "consumes" an element and won't be available for other consumers. For example a job queue. Also typically queues either grow unbounded or pushes fail. So no, I wouldn't say that the disruptor is a normal SPMC or MPMC queue as it semantics are different. But yes, it has pretty much the same characteristics of UDP.

I see what you mean. I would never use a pattern where you don't broadcast to all consumers myself.

And while you can fail to produce when a consumer is too slow, I'd argue that sort of pattern only makes sense for a single consumer as well (the network equivalent would be TCP, which by definition can only be unicast).

Re: LMAX Disruptor – High Performance Inter-Thread Messaging Library

#87

Post by samsquire »

cplusplusfellow wrote: post #12
samsquire wrote: post #2

I am working on a C version of the disruptor ringbuffer it is very simple and I need to verify it so it's probably not ready for others but it might be interesting. Aligning by 128 bytes has dropped latency and stopped false sharing. I have gotten latencies to 50 nanoseconds and up. disruptor-multi.c(SPMC) and disruptor-multi-producer.c (MPSC) https://GitHub.com/samsquire/assembly I am trying to work out how to suppo…

Do you think it’s possible to obtain this performance with Rust? I’ve been down the path you’re on a few times and I love the pursuit. Have built my own over the years about 4 times. Hardware was much slower in those days so my lower barrier was 650ns. Things got worse appreciably as a function of the number of producers I found. Some of my most sleepless nights. The funnest nights.

How many producers and how many consumers is that 650 nanoseconds?

I have pinned threads to even numbered cores with pthread_setaffinity_np and that seems to have evened out the MPMC ringbuffer - 2 producers 2 consumers to under 400 nanoseconds, usually under 1000 nanoseconds. I think hyperthreading causes problems.

EDIT: Would you like to chat about this? I would like to! My email is in my profile.

Re: LMAX Disruptor – High Performance Inter-Thread Messaging Library

#88

Post by silasb »

dajt wrote: post #39
onetimeuse92304 wrote: post #33

Earlier quoted context omitted.

Machine code isn't some kind of magical idea. It is just bytes and you can craft the bytes any way you want. In my case I wrote my own compiler that generated machine code bytes based on my needs. The compiler wasn't at all complicated. It was not general compiler that would have to be able to deal with everything you could throw at it. It only supported one model of CPU. It did not require any optimisation mechanism…

How did you inject the machine code into a running program on anything near a modern architecture? Unless you had your own OS, wouldn't code segments be RO and data segments not executable? I don't know enough about any processor created in the last 30 years to know if running bare metal without a commercial OS would allow you to not have those constraints.

Can't you use `mprotect` to allow you to write to memory and execute?

https://github.com/Frodox/execute-machine-code-from-memory/t... has several examples of how to do it.

EDIT: https://www.kvakil.me/posts/2022-10-13-optimizing-mprotect-i... is another good link that explains this.

Re: LMAX Disruptor – High Performance Inter-Thread Messaging Library

#89

Post by gpderetta »

mgaunard wrote: post #86
gpderetta wrote: post #83

Earlier quoted context omitted.

Topically in an MC queue, a consumer "consumes" an element and won't be available for other consumers. For example a job queue. Also typically queues either grow unbounded or pushes fail. So no, I wouldn't say that the disruptor is a normal SPMC or MPMC queue as it semantics are different. But yes, it has pretty much the same characteristics of UDP.

I see what you mean. I would never use a pattern where you don't broadcast to all consumers myself. And while you can fail to produce when a consumer is too slow, I'd argue that sort of pattern only makes sense for a single consumer as well (the network equivalent would be TCP, which by definition can only be unicast).

Well, TCP is stream oriented and unicast, but doesn't take much to conceive a message oriented protocol that does anycasting.

As a real world example of anycasting queue that I'm sure you have used, consider the queue behind accept(2) or pthread_condition_wait.

Post reply on HN