Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ultra-low-latency, batching and concurrent queue for IPC in Java

github.com

Post reply on HN
Print view View original

41–50 of 76 posts

Re: Ultra-low-latency, batching and concurrent queue for IPC in Java

#41

Post by joas_coder »

bsder wrote: post #7
WeaselNo7 wrote: post #2

Weird to see this here! I've used CoralBlocks in the low-latency trading domain previously. Highly recommend. The API is kind, they're very responsive, and the latency is exceptional (and comes with all the basics like thread pinning built-in for convenience)

Given the documentation says that this is supposedly to be between JVMs, how do they handle the serialize/deserialize?

> Given the documentation says that this is supposedly to be between JVMs, how do they handle the serialize/deserialize?

Your transfer object needs to implement MemorySerializable. Below two examples from CoralRing's GitHub:

https://github.com/coralblocks/CoralRing/blob/main/src/main/...

https://github.com/coralblocks/CoralRing/blob/main/src/main/...

The second one effectively allows you to send anything you want (as bytes) through the ring, making CoralRing message agnostic.

Re: Ultra-low-latency, batching and concurrent queue for IPC in Java

#42

Post by joas_coder »

defer wrote: post #26
nwellinghoff wrote: post #22

I don’t get it. How is this advantageous as it’s limited to one machine? Why wouldn’t you just have one jvm running multiple threads? What is the point of having multiple jvm processes interacting through this ring? Can someone enlighten me?

JVM does garbage collection, this can stop all threads at safepoints while GC occurs. Those stops can be enough to ruin your low latency requirements in the high percentiles. A common strategy is to divide workloads between jvms so that you meet the requirement.

CoralRing and CoralQueue (available on GitHub) are completely garbage-free. You can send billions of messages without ever creating garbage for the GC, so no GC overhead. This is paramount for real-time ultra-low-latency systems developed in Java. You can read more about it here => https://www.coralblocks.com/index.php/java-development-witho...

Re: Ultra-low-latency, batching and concurrent queue for IPC in Java

#43

Post by joas_coder »

sylware wrote: post #35

ultra-low-latency and java in the same sentence... masochists?

LOL you have a point. But this is subjective. Someone could argue that a highly complex C++ system is masochism. The reality is: a lot of successful market makers, prop trading firms, banks, etc. use Java for ultra-low-latency financial systems. And of course some successful companies use C++ as well.

Re: Ultra-low-latency, batching and concurrent queue for IPC in Java

#44

Post by speed_spread »

sylware wrote: post #35

ultra-low-latency and java in the same sentence... masochists?

Not at all. The constraints for ultra low latency are the same whether you use C++ or Java: No blocking IO, no dynamic memory allocation. Using the right libraries such as this one, Java can do a surprisingly decent job while keeping the code more accessible to a layman developer than if it was C++. You still need C++ for applications where maximum mechanical sympathy is required (cache alignment guarantees, etc.) but if you have low latency / medium volume Java often does the trick.

Re: Ultra-low-latency, batching and concurrent queue for IPC in Java

#45

Post by lizhenqi »

capmorganbih wrote: post #12
sriram_malhar wrote: post #9

Earlier quoted context omitted.

Can you explain what your issue is with Kafka? What makes it not reliable enough?

Kafka is good. The problem is we don’t have a dedicated person to manage it so sometimes we have kafka outages

Just a suggestion, you can learn about automq (a message queue that has rewritten the storage layer based on Kafka), which has done a lot in automatic recovery and self-balancing, maybe it will help you. If you don't need it, you can ignore this comment.

Re: Ultra-low-latency, batching and concurrent queue for IPC in Java

#46

Post by joas_coder »

speed_spread wrote: post #44
sylware wrote: post #35

ultra-low-latency and java in the same sentence... masochists?

Not at all. The constraints for ultra low latency are the same whether you use C++ or Java: No blocking IO, no dynamic memory allocation. Using the right libraries such as this one, Java can do a surprisingly decent job while keeping the code more accessible to a layman developer than if it was C++. You still need C++ for applications where maximum mechanical sympathy is required (cache alignment guarantees, etc.) bu…

Well said and I couldn't agree more. There are top market makers and banks using Java for a fact. And other C++ firms as well. Some of them are considering or have considered the move to Java. Some have already done this move. Some will never do. I'm certain that it is trivial for a C++ programmer to code in Java. The opposite of course is not true. The whole point of Java as a language is to be higher-level than C++. I don't know if people have realized, but with GraalVM it is now possible to compile Java code entirely to native code ahead-of-time, like it is done with C++. Even before GraalVM there was already the -Xcomp option to force JIT compilation in the very first pass. However that does not necessarily mean that AOT is always preferable over JIT. It is not. Runtime profiling information so you can determine the critical path (hot spots) is amazing for some optimizations, such as aggressive inlining.

Re: Ultra-low-latency, batching and concurrent queue for IPC in Java

#47

Post by kookamamie »

joas_coder wrote: post #43
sylware wrote: post #35

ultra-low-latency and java in the same sentence... masochists?

LOL you have a point. But this is subjective. Someone could argue that a highly complex C++ system is masochism. The reality is: a lot of successful market makers, prop trading firms, banks, etc. use Java for ultra-low-latency financial systems. And of course some successful companies use C++ as well.

Numbers aren't subjective. High-performance is a measurable claim and shouldn't be entangled with language-complexity matters.

Re: Ultra-low-latency, batching and concurrent queue for IPC in Java

#48

Post by kasey_junk »

gunnarmorling wrote: post #25
nwellinghoff wrote: post #22

I don’t get it. How is this advantageous as it’s limited to one machine? Why wouldn’t you just have one jvm running multiple threads? What is the point of having multiple jvm processes interacting through this ring? Can someone enlighten me?

A few potential reasons for this design coming to mind: - Resource allocation; you might want to give just specific amount of memory, CPU, network I/O to specific modules of a system, which is not really feasible within a single JVM - Resource isolation; e.g. a memory leak in one module of the system will affect just that specific JVM instance but not others (similar to why browsers run tabs in multiple processes); -…

Also software design. You can split jvm into those that have to follow strict parameters (eg no allocations) and those that follow more traditional Java patterns.

Re: Ultra-low-latency, batching and concurrent queue for IPC in Java

#49

Post by joas_coder »

kookamamie wrote: post #47
joas_coder wrote: post #43

Earlier quoted context omitted.

LOL you have a point. But this is subjective. Someone could argue that a highly complex C++ system is masochism. The reality is: a lot of successful market makers, prop trading firms, banks, etc. use Java for ultra-low-latency financial systems. And of course some successful companies use C++ as well.

Numbers aren't subjective. High-performance is a measurable claim and shouldn't be entangled with language-complexity matters.

Totally agree! My understanding because of the use of the word "masochist" was that it was possible but difficult. Not that one language was better/worse than the other, in terms of performance.

Re: Ultra-low-latency, batching and concurrent queue for IPC in Java

#50

Post by joas_coder »

nwellinghoff wrote: post #27
defer wrote: post #26

Earlier quoted context omitted.

JVM does garbage collection, this can stop all threads at safepoints while GC occurs. Those stops can be enough to ruin your low latency requirements in the high percentiles. A common strategy is to divide workloads between jvms so that you meet the requirement.

Interesting. But how do you ensure a worker that picks up a task does not pause on gc as well?

CoralRing does not produce garbage, but it cannot control what other parts of your application choose to do. It will hand to your application a message without producing any garbage, now if you go ahead and produce garbage yourself then there is nothing CoralRing can do about that. Ultra-low-latency applications in Java are designed so that nothing in the critical path produces garbage.
Post reply on HN