Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ultra-low-latency, batching and concurrent queue for IPC in Java

github.com

Post reply on HN
Print view View original

71–76 of 76 posts

Re: Ultra-low-latency, batching and concurrent queue for IPC in Java

#71

Post by nulld3v »

joas_coder wrote: post #39
nulld3v wrote: post #36

Earlier quoted context omitted.

It looks like these are more lower level libraries so clients are supposed to block, but they are responsible for doing the blocking themselves. And given these are designed for "ultra-low-latency" systems, I don't think that's a big problem because the best blocking strategy is probably just to spin. It would be nice if the docs were nicer though, considering this is a paid product...

> It looks like these are more lower level libraries so clients are supposed to block, but they are responsible for doing the blocking themselves. You busy spin when blocking (fastest) or you can use a WaitStrategy from https://www.github.com/coralblocks/CoralQueue . You can see an example here: https://github.com/coralblocks/CoralQueue/blob/main/src/main... > And given these are designed for "ultra-low-latency" syst…

Oof, I saw "CoralQueue" on the product list on your website and I just immediately assumed it was a paid product. But I didn't even think of looking at the LICENSE files, my bad!

Regarding the docs, I was only looking at CoralRing, and it looks like CoralQueue has some additional documentation that applies to CoralRing as well. After reading through them everything makes a lot more sense.

Re: Ultra-low-latency, batching and concurrent queue for IPC in Java

#72

Post by kllrnohj »

joas_coder wrote: post #38
kllrnohj wrote: post #16

https://github.com/coralblocks/CoralRing/blob/1168b047e0183c... Am I missing something here or does the BlockingRingConsumer not actually block? And worse doesn't it just return garbage if poll is called without first checking availableToPoll? The example sure looks like it... https://github.com/coralblocks/CoralRing/blob/main/src/main/... Which if so isn't this like 1/4th a library for doing IPC? It doesn't seem to…

availableToPoll() is the method that blocks, because it can return 0 on an empty ring. When that happens you have to block, by for example busy spinning around availableToPoll() until it returns something to poll. You are never supposed to call poll() without first calling availableToPoll(). How can you poll something if you don't know if there is something available to poll? This is very different than a ConcurrentL…

> availableToPoll() is the method that blocks, because it can return 0 on an empty ring.

Which means it doesn't block...

Blocking means the call itself doesn't return until there's data available. This is a completely non-blocking API. Which is fine, it's just very wrongly named in that case.

But that also means I don't really know why there's both "non-blocking" and "blocking" variants at that point if blocking isn't an option at all in the first place.

Re: Ultra-low-latency, batching and concurrent queue for IPC in Java

#73

Post by sylware »

speed_spread wrote: post #44
sylware wrote: post #35

ultra-low-latency and java in the same sentence... masochists?

Not at all. The constraints for ultra low latency are the same whether you use C++ or Java: No blocking IO, no dynamic memory allocation. Using the right libraries such as this one, Java can do a surprisingly decent job while keeping the code more accessible to a layman developer than if it was C++. You still need C++ for applications where maximum mechanical sympathy is required (cache alignment guarantees, etc.) bu…

I do it better, I free myself from as much as I can of toxic SDK (c++ and java SDKs are filthy toxic).

Assembly, worst case scenario plain and simple C99+.

Re: Ultra-low-latency, batching and concurrent queue for IPC in Java

#74

Post by sylware »

speed_spread wrote: post #67
blibble wrote: post #61

Earlier quoted context omitted.

dynamic allocation is fine as long as it is under control, as it's just a pointer bump what you never want is to trigger the GC at a bad time (you can do cache alignment too relatively easily)

Doesn't dynamic allocation expose you to memory fragmentation or other non-deterministic phenomenons?

With semantic and fine-grained control of your allocations, and with a memory paging system on top of that, getting fragmentation is tough...

Re: Ultra-low-latency, batching and concurrent queue for IPC in Java

#75

Post by joas_coder »

kllrnohj wrote: post #72
joas_coder wrote: post #38

Earlier quoted context omitted.

availableToPoll() is the method that blocks, because it can return 0 on an empty ring. When that happens you have to block, by for example busy spinning around availableToPoll() until it returns something to poll. You are never supposed to call poll() without first calling availableToPoll(). How can you poll something if you don't know if there is something available to poll? This is very different than a ConcurrentL…

> availableToPoll() is the method that blocks, because it can return 0 on an empty ring. Which means it doesn't block... Blocking means the call itself doesn't return until there's data available. This is a completely non-blocking API. Which is fine, it's just very wrongly named in that case. But that also means I don't really know why there's both "non-blocking" and "blocking" variants at that point if blocking isn'…

You are going to block/wait yourself when availableToPoll() (now changed to availableToFetch() for clarity) returns 0. When that happens you can block/wait by busy spinning or by using a wait strategy. The blocking term means that the producer has to block/wait when the ring is full and the consumer has to block/wait when the ring is empty.

For the non-blocking ring, the producer never blocks on a full ring. It overwrites the circular ring. The consumer can still block on an empty ring.

So to make it clear:

Blocking ring => producer and consumer can block/wait

Non-Blocking ring => producer never blocks and consumer can still block on an empty ring. Consumer can also fall behind too much and disconnect.

Re: Ultra-low-latency, batching and concurrent queue for IPC in Java

#76

Post by joas_coder »

algo_trader wrote: post #5
WeaselNo7 wrote: post #2

Weird to see this here! I've used CoralBlocks in the low-latency trading domain previously. Highly recommend. The API is kind, they're very responsive, and the latency is exceptional (and comes with all the basics like thread pinning built-in for convenience)

> CoralBlocks in the low-latency trading domain previously. Yeah, modern JVM is a true miracle and you can be x5 productive (and safe!) compared to C/C++ Do you have any recommendations for a low latency work queue (with in a jvm)? I want to spawn millions of micro-second-tasks per second, to worker cores.. I am on a massive cache CPU so memory latency hasnt raised its ugly head yet EDIT: not LMAX please...

You were interested in the diamond queue, so just wanted to say that it is now released in our GitHub => https://github.com/coralblocks/CoralQueue?tab=readme-ov-file...
Post reply on HN