Earlier quoted context omitted.
You could but its not purpose built for that. You'd probably be happier using some other memory mapped file format for that.
Could you please suggest java library for this if you know one?
crazy idea, but SQLite
Ultra-low-latency, batching and concurrent queue for IPC in Java
21–30 of 76 posts
Re: Ultra-low-latency, batching and concurrent queue for IPC in Java
#22I don’t get it. How is this advantageous as it’s limited to one machine? Why wouldn’t you just have one jvm running multiple threads? What is the point of having multiple jvm processes interacting through this ring? Can someone enlighten me?
Re: Ultra-low-latency, batching and concurrent queue for IPC in Java
#23Earlier quoted context omitted.
Can you explain what your issue is with Kafka? What makes it not reliable enough?
Kafka is good. The problem is we don’t have a dedicated person to manage it so sometimes we have kafka outages
So basically, you want to build a write-ahead-log before writing data to Kafka, and I think you're underestimating the effort to implement a WAL.
If you don't have a person that can manage Kafka, you almost definitely don't have the person to maintain a WAL.
Re: Ultra-low-latency, batching and concurrent queue for IPC in Java
#24Related: https://github.com/pcdv/jocket
Drop-in replacement for java.net.Socket using shared memory (and optionally, futex for notification)
Re: Ultra-low-latency, batching and concurrent queue for IPC in Java
#25I don’t get it. How is this advantageous as it’s limited to one machine? Why wouldn’t you just have one jvm running multiple threads? What is the point of having multiple jvm processes interacting through this ring? Can someone enlighten me?
A few potential reasons for this design coming to mind:
- Resource allocation; you might want to give just specific amount of memory, CPU, network I/O to specific modules of a system, which is not really feasible within a single JVM
- Resource isolation; e.g. a memory leak in one module of the system will affect just that specific JVM instance but not others (similar to why browsers run tabs in multiple processes);
- Upgrades; you can put a new version of one module of the system into place without impacting the others; while the JVM does support this via dynamic classloading (as e.g. used in OSGi or Layrry, https://github.com/moditect/layrry), this becomes complex quickly, you can create classloader leaks, etc.
- Security; You might have (3rd-party) modules you want to keep isolated from the memory, data, config, etc. of other modules; in particular with the removal of the security manager, OS-enforced process isolation is the way to
Re: Ultra-low-latency, batching and concurrent queue for IPC in Java
#26I don’t get it. How is this advantageous as it’s limited to one machine? Why wouldn’t you just have one jvm running multiple threads? What is the point of having multiple jvm processes interacting through this ring? Can someone enlighten me?
JVM does garbage collection, this can stop all threads at safepoints while GC occurs.
Those stops can be enough to ruin your low latency requirements in the high percentiles. A common strategy is to divide workloads between jvms so that you meet the requirement.
Re: Ultra-low-latency, batching and concurrent queue for IPC in Java
#27I don’t get it. How is this advantageous as it’s limited to one machine? Why wouldn’t you just have one jvm running multiple threads? What is the point of having multiple jvm processes interacting through this ring? Can someone enlighten me?
JVM does garbage collection, this can stop all threads at safepoints while GC occurs. Those stops can be enough to ruin your low latency requirements in the high percentiles. A common strategy is to divide workloads between jvms so that you meet the requirement.
Interesting. But how do you ensure a worker that picks up a task does not pause on gc as well?
Re: Ultra-low-latency, batching and concurrent queue for IPC in Java
#28I don’t get it. How is this advantageous as it’s limited to one machine? Why wouldn’t you just have one jvm running multiple threads? What is the point of having multiple jvm processes interacting through this ring? Can someone enlighten me?
I can throw some guesses: 1) apps deployed in separate Docker containers due to organization's tech team separation, 2) apps that require security/performance isolation among tenants, 3) isolation layer around memory-leaky and bug-prone third-party library code.
Re: Ultra-low-latency, batching and concurrent queue for IPC in Java
#29Earlier quoted context omitted.
JVM does garbage collection, this can stop all threads at safepoints while GC occurs. Those stops can be enough to ruin your low latency requirements in the high percentiles. A common strategy is to divide workloads between jvms so that you meet the requirement.
Interesting. But how do you ensure a worker that picks up a task does not pause on gc as well?
Maybe they run a small heap with a zero-pause JVM like Zing, as pause-less GC generally has lower throughput than normal GC.
Re: Ultra-low-latency, batching and concurrent queue for IPC in Java
#30Weird to see this here! I've used CoralBlocks in the low-latency trading domain previously. Highly recommend. The API is kind, they're very responsive, and the latency is exceptional (and comes with all the basics like thread pinning built-in for convenience)
> CoralBlocks in the low-latency trading domain previously. Yeah, modern JVM is a true miracle and you can be x5 productive (and safe!) compared to C/C++ Do you have any recommendations for a low latency work queue (with in a jvm)? I want to spawn millions of micro-second-tasks per second, to worker cores.. I am on a massive cache CPU so memory latency hasnt raised its ugly head yet EDIT: not LMAX please...
>any recommendations for a low latency work queue (with in a jvm)?
I toyed around the ring buffer pattern a decade ago, creating a unicast one (using CAS on entries, and eventually a logarithmic scan for next readable entry, not to brute-force-scan them all), but I'm not sure that its latency is much better than that of a regular ThreadPoolExecutor (the throughput could be better though).
Latency also depends on whether it spins or blocks when waiting for a slot to read or write.
If you want to give it a try: https://github.com/jeffhain/jodk/blob/master/src/net/jodk/th...