Ultra-low-latency, batching and concurrent queue for IPC in Java • Hacker News Board

Re: Ultra-low-latency, batching and concurrent queue for IPC in Java

#51

Post by gpderetta » Thu, Nov 28, 2024, 2:52 PM UTC

joas_coder wrote: post #38

kllrnohj wrote: post #16

https://github.com/coralblocks/CoralRing/blob/1168b047e0183c... Am I missing something here or does the BlockingRingConsumer not actually block? And worse doesn't it just return garbage if poll is called without first checking availableToPoll? The example sure looks like it... https://github.com/coralblocks/CoralRing/blob/main/src/main/... Which if so isn't this like 1/4th a library for doing IPC? It doesn't seem to…

availableToPoll() is the method that blocks, because it can return 0 on an empty ring. When that happens you have to block, by for example busy spinning around availableToPoll() until it returns something to poll. You are never supposed to call poll() without first calling availableToPoll(). How can you poll something if you don't know if there is something available to poll? This is very different than a ConcurrentL…

> How can you poll something if you don't know if there is something available to poll

counterexample: you can call ::poll(2) if there isn't anything to available. It will either block or return as desired. That's literally what I would expect a poll function to do: check if something is available; c.f. busy-polling.

Maybe it is a language difference?

Re: Ultra-low-latency, batching and concurrent queue for IPC in Java

#52

Post by 1oooqooq » Thu, Nov 28, 2024, 3:11 PM UTC

it's sad how comfy the world for spawning request is versus when you are on the receiving side.

Re: Ultra-low-latency, batching and concurrent queue for IPC in Java

#53

Post by joas_coder » Thu, Nov 28, 2024, 3:12 PM UTC

gpderetta wrote: post #51

joas_coder wrote: post #38

Earlier quoted context omitted.

availableToPoll() is the method that blocks, because it can return 0 on an empty ring. When that happens you have to block, by for example busy spinning around availableToPoll() until it returns something to poll. You are never supposed to call poll() without first calling availableToPoll(). How can you poll something if you don't know if there is something available to poll? This is very different than a ConcurrentL…

> How can you poll something if you don't know if there is something available to poll counterexample: you can call ::poll(2) if there isn't anything to available. It will either block or return as desired. That's literally what I would expect a poll function to do: check if something is available; c.f. busy-polling. Maybe it is a language difference?

Not a language difference, but a design difference. By calling availableToPoll before polling you can poll a batch of messages without blocking or locking on each message. This design decision allows the ring to perform batching naturally. And batching is very important for perfomance. java.util.concurrent.ConcurrentLinkedQueue does not support batching.

Re: Ultra-low-latency, batching and concurrent queue for IPC in Java

#54

Post by gpderetta » Thu, Nov 28, 2024, 3:23 PM UTC

dikei wrote: post #34

gunnarmorling wrote: post #25

Earlier quoted context omitted.

A few potential reasons for this design coming to mind: - Resource allocation; you might want to give just specific amount of memory, CPU, network I/O to specific modules of a system, which is not really feasible within a single JVM - Resource isolation; e.g. a memory leak in one module of the system will affect just that specific JVM instance but not others (similar to why browsers run tabs in multiple processes); -…

Yeah, I think the HFT guys use CPU pinning a lot: 1 process - 1 CPU, so you'd need multiple processes to take advantage of multicores server.

Usually it is 1 thread - 1 CPU. There might be other reasons (address space separation has its own advantages - and disadvantages) to have distinct processes of course.

Re: Ultra-low-latency, batching and concurrent queue for IPC in Java

#55

Post by pebal » Thu, Nov 28, 2024, 3:45 PM UTC

dikei wrote: post #33

pebal wrote: post #31

Earlier quoted context omitted.

Java doesn't have real pause-less GC.

Well, "pause too short to matter" just doesn't have the same ring to it.

One millisecond is not a short pause.

Re: Ultra-low-latency, batching and concurrent queue for IPC in Java

#56

Post by knome » Thu, Nov 28, 2024, 3:59 PM UTC

joas_coder wrote: post #53

gpderetta wrote: post #51

Earlier quoted context omitted.

> How can you poll something if you don't know if there is something available to poll counterexample: you can call ::poll(2) if there isn't anything to available. It will either block or return as desired. That's literally what I would expect a poll function to do: check if something is available; c.f. busy-polling. Maybe it is a language difference?

Not a language difference, but a design difference. By calling availableToPoll before polling you can poll a batch of messages without blocking or locking on each message. This design decision allows the ring to perform batching naturally. And batching is very important for perfomance. java.util.concurrent.ConcurrentLinkedQueue does not support batching.

I don't think anyone is questioning the strategy, just the naming.

Polling, as a term, already generally means asking if there is anything ready. The library might instead have had a poll() to indicate readiness and an unsafeTakeReadyBatch()/releaseBatch() or similar to handle the low level receipt primitives. Or you could even have had a checkForReady() and left poll() to implement a generic version of your busy waiting example, polling with a given sleep and timeout until items were ready, as a convenience for the user.

It's fine as is, of course, as you document expected usage quite well.

Re: Ultra-low-latency, batching and concurrent queue for IPC in Java

#57

Post by joas_coder » Thu, Nov 28, 2024, 4:21 PM UTC

knome wrote: post #56

joas_coder wrote: post #53

Earlier quoted context omitted.

Not a language difference, but a design difference. By calling availableToPoll before polling you can poll a batch of messages without blocking or locking on each message. This design decision allows the ring to perform batching naturally. And batching is very important for perfomance. java.util.concurrent.ConcurrentLinkedQueue does not support batching.

I don't think anyone is questioning the strategy, just the naming. Polling, as a term, already generally means asking if there is anything ready. The library might instead have had a poll() to indicate readiness and an unsafeTakeReadyBatch()/releaseBatch() or similar to handle the low level receipt primitives. Or you could even have had a checkForReady() and left poll() to implement a generic version of your busy wai…

Oh, I see now. I thought he was talking about a computer language but he was actually talking about the English language. My bad.

So are you saying that POLL was a bad choice of name because polling means "check if there is something available and if there is get it"?

What would be a better name? How about availableToFetch and fetch? Any other better idea?

I don't like remove because you are not actually removing the object from the ring.

Re: Ultra-low-latency, batching and concurrent queue for IPC in Java

#58

Post by gpderetta » Thu, Nov 28, 2024, 4:30 PM UTC

joas_coder wrote: post #57

knome wrote: post #56

Earlier quoted context omitted.

I don't think anyone is questioning the strategy, just the naming. Polling, as a term, already generally means asking if there is anything ready. The library might instead have had a poll() to indicate readiness and an unsafeTakeReadyBatch()/releaseBatch() or similar to handle the low level receipt primitives. Or you could even have had a checkForReady() and left poll() to implement a generic version of your busy wai…

Oh, I see now. I thought he was talking about a computer language but he was actually talking about the English language. My bad. So are you saying that POLL was a bad choice of name because polling means "check if there is something available and if there is get it"? What would be a better name? How about availableToFetch and fetch? Any other better idea? I don't like remove because you are not actually removing the…

poll doesn't necessarily imply removing.

Re: Ultra-low-latency, batching and concurrent queue for IPC in Java

#59

Post by joas_coder » Thu, Nov 28, 2024, 4:43 PM UTC

gpderetta wrote: post #58

joas_coder wrote: post #57

Earlier quoted context omitted.

Oh, I see now. I thought he was talking about a computer language but he was actually talking about the English language. My bad. So are you saying that POLL was a bad choice of name because polling means "check if there is something available and if there is get it"? What would be a better name? How about availableToFetch and fetch? Any other better idea? I don't like remove because you are not actually removing the…

poll doesn't necessarily imply removing.

It is clear to me now that poll only removes if there is something to remove

A much better name would have been FETCH and not POLL

We'll be changing everything from poll to fetch in the next version => availableToFetch() and fetch()

Re: Ultra-low-latency, batching and concurrent queue for IPC in Java

#60

Post by gpderetta » Thu, Nov 28, 2024, 4:51 PM UTC

joas_coder wrote: post #59

gpderetta wrote: post #58

Earlier quoted context omitted.

poll doesn't necessarily imply removing.

It is clear to me now that poll only removes if there is something to remove A much better name would have been FETCH and not POLL We'll be changing everything from poll to fetch in the next version => availableToFetch() and fetch()

While we are bikeshedding, what about peek() (returns the element, but doesn't dequeue), pop() (dequeue and return the element), try_peek(), try_pop() for the polling variants.