Live data from Hacker News

Why low-latency Java still requires discipline?

chronicle.software

51–56 of 56 posts

Re: Why low-latency Java still requires discipline?

#51

Earlier quoted context omitted.

In heavily performance-engineered code having a GC coalesce memory is a pessimization in all cases. I've done a lot of performance engineering in both C++ and Java. Every optimization available in Java also exists in C++ but the reverse is not true, which is why C++ is always faster. Every example I have ever seen of Java being faster than C++ was just poorly optimized code. The heuristic I use is that heavily optimi…

What type and size of applications have you worked on? @pron, who works on the JVM and is a C++ expert, has been writing a lot recently about low level languages becoming increasingly impossible to optimize with more LOC and more people working on it (dozens, or even hundreds of developers). The idea being be a language with an aggressive JIT, moving garbage collector, and bump allocation is going to allocate/dealloc…

I work on analytical data engines. Lots of spatial and graph-like joins on PB-to-EB scale data models often concurrent with extremely high rates of real-time ingest into the same data model. I’ve also done a lot of work in HPC and more conventional databases at smaller scales. Typical code base is in the range of 100k-1M LoC in C++. Java has more LoC equivalent but it isn’t used much for these applications anymore because it scales significantly worse as systems and hardware become larger.

Almost all optimization is architectural in nature. Properly performance-engineered code allocates no memory after bootstrap. There is no possible way even in theory for a GC to outperform schedule-aware allocation from a fixed pool. In most cases the GC is just generic C++ code anyway; that indirection is unnecessary.

An important caveat is that I mostly deal with throughput-optimized code. It isn’t latency-sensitive and this article is about low-latency code. Nonetheless, the schedulers require predictable execution at ~1µs granularity for throughput optimization purposes, which is difficult to guarantee in Java.

I can write highly optimized code in C++ in a straightforward way that doesn’t really have a Java equivalent because equivalent guarantees are not provided as a practical matter.

TBH, even when I wasn’t trying to performance-engineer code I’ve never seen Java run as fast as the equivalent C++.

Re: Why low-latency Java still requires discipline?

#52
post #35

Earlier quoted context omitted.

First of all it depends on the language, what GC models it supports, and what are the performance SLAs. Secondly if we are talking about something like real time audio, or sending the maximum amount of triangles per second, then there is zero memory management, the code should be allocation free across all the critical path, regardless of the language.

Not just allocation free. It needs to be page-in free too. And critical loops need to be cacheline friendly.

Yeah, that as well.

Re: Why low-latency Java still requires discipline?

#53

Earlier quoted context omitted.

I need a UI which runs well on Windows, MacOS, and Linux, without having to build three different ones. Swing is still easily the best, most consistent, and most native-feeling cross-platform environment. It's much better than QT and GTK in most respects. And Java also runs elegantly on a little platform you may know as Android. I have high hopes for go and rust. But until they have mature UIs, they're out (for me).…

In heavily performance-engineered code having a GC coalesce memory is a pessimization in all cases. I've done a lot of performance engineering in both C++ and Java. Every optimization available in Java also exists in C++ but the reverse is not true, which is why C++ is always faster. Every example I have ever seen of Java being faster than C++ was just poorly optimized code. The heuristic I use is that heavily optimi…

> which is why C++ is always faster

always CAN be given one is a sufficiently experienced C++ developer, with sufficient time and effort (both for the first iteration, and for every future maintenance).

The stars rarely align well on all counts and one may very well be better off with a close-enough, cheaper Java version.

Re: Why low-latency Java still requires discipline?

#54

Earlier quoted context omitted.

What type and size of applications have you worked on? @pron, who works on the JVM and is a C++ expert, has been writing a lot recently about low level languages becoming increasingly impossible to optimize with more LOC and more people working on it (dozens, or even hundreds of developers). The idea being be a language with an aggressive JIT, moving garbage collector, and bump allocation is going to allocate/dealloc…

I work on analytical data engines. Lots of spatial and graph-like joins on PB-to-EB scale data models often concurrent with extremely high rates of real-time ingest into the same data model. I’ve also done a lot of work in HPC and more conventional databases at smaller scales. Typical code base is in the range of 100k-1M LoC in C++. Java has more LoC equivalent but it isn’t used much for these applications anymore be…

> There is no possible way even in theory for a GC to outperform schedule-aware allocation from a fixed pool.

You mean in the absence of any reference-counting overhead? That doesn't sound like a fair comparison.

> In most cases the GC is just generic C++ code anyway; that indirection is unnecessary.

You can't use the JVM's GCs from C++. C++ heavily relies on RAII, so it doesn't seem like a good fit anyway. I'm not aware of any serious efforts at a high performance GC for C++.

As Herb Sutter points out, moving GCs require non-stable pointers, which C++ isn't suited to. [0]

> the schedulers require predictable execution at ~1µs granularity for throughput optimization purposes, which is difficult to guarantee in Java

Can't say I know much about that kind of issue, how does it work in C++ code?

[0] https://herbsutter.com/2011/10/25/garbage-collection-synopsi...

Re: Why low-latency Java still requires discipline?

#55

Earlier quoted context omitted.

In heavily performance-engineered code having a GC coalesce memory is a pessimization in all cases. I've done a lot of performance engineering in both C++ and Java. Every optimization available in Java also exists in C++ but the reverse is not true, which is why C++ is always faster. Every example I have ever seen of Java being faster than C++ was just poorly optimized code. The heuristic I use is that heavily optimi…

What type and size of applications have you worked on? @pron, who works on the JVM and is a C++ expert, has been writing a lot recently about low level languages becoming increasingly impossible to optimize with more LOC and more people working on it (dozens, or even hundreds of developers). The idea being be a language with an aggressive JIT, moving garbage collector, and bump allocation is going to allocate/dealloc…

I recommend his recent appearance on the Inside Java podcast, where these topics are explored well. The episode is number 59, titled Java *is* Memory Efficient.

https://www.youtube.com/watch?v=M_HCG1JPMQE (or any podcast application)

Re: Why low-latency Java still requires discipline?

#56
Chronicle has a lot of great resources. I’ve been out of Java for a few years but when I had to write high perf code, the libraries and blog insights were invaluable.

A lot of the advice is good in general - keeping things simple, generating fewer objects, etc. Profiling with Yourkit or JMH to find and improve slow spots.

This let us build high performance software that had a vast difference in say p90 input size and p99. It involved rewriting third party libraries to let us 3x speed and substantially reduce GC. I think in the end we were around a p99 of 7ms and p90 well under that (the data spread was kbs to megabytes) across hundreds of millions of inputs per day.

Post reply on HN