Had to write a router software once in Java back in 2014. The past experience in me believed that its not a good choice to use something with stop the world GCs for routers. The goal was to have everything under 1ms (max 2ms in rare cases). It has http server on head and websocket connections on the tail end. Turns out the class instantiations and JSON parsing were a bottleneck. Thread safe resources were valuable, the libraries which did not had thread safe resources were re-written to support what we needed. The http server was re-written barebones, the JSON parsing was re-written with barebones. And it worked.
Why low-latency Java still requires discipline?
31–40 of 56 posts
Re: Why low-latency Java still requires discipline?
#32I get that blog posts often advertise a company's products, but this one had absolutely zero content other than advertising.
I think it said something very meaningful and actionable (admittedly at a high level) about keeping the hot path deterministic: low allocations (including guidelines for GC frequency targets), don't use vthreads (instead use one thread on a pcore that never sleeps, to avoid amdahl), caution about techiques that work at p99 and fail at 99.999... This is not typical for "enterprise" Java, and is an interesting article for those who wish their systems were more deterministic (but may not be aware of the techniques or their cost). This is not a "how to", but that's fine. It's still good content.
Re: Why low-latency Java still requires discipline?
#33How about not using Java? Then you can have low latency. Average go, rust, c++ and c will outperform amazing java programs, and the former will also be way way more easy to run, troubleshoot, interpret logs from. Java is usch garbage in every stack.
I've done some tests with current value classes in latest prototype with full optimisation through annotation. Apparently, everything was being compiled to assembly code with no gc calls. It seems this old school ideology of slow java is about to end in near future. I'm actually intrigued how it will compete with Rust which can't optimise code further while running, while JVM JIT has more info which it could aggressively further optimise, especially hot paths.
Re: Why low-latency Java still requires discipline?
#34Even C requires discipline to write low latency code, if you think otherwise, you never used a profiler.
Yes. & in the end my experience has been that you can write low latency code with gc, but only by expending more effort than it'd take to use manual memory management
Re: Why low-latency Java still requires discipline?
#35Even C requires discipline to write low latency code, if you think otherwise, you never used a profiler.
Yes. & in the end my experience has been that you can write low latency code with gc, but only by expending more effort than it'd take to use manual memory management
First of all it depends on the language, what GC models it supports, and what are the performance SLAs.
Secondly if we are talking about something like real time audio, or sending the maximum amount of triangles per second, then there is zero memory management, the code should be allocation free across all the critical path, regardless of the language.
Re: Why low-latency Java still requires discipline?
#36I get that blog posts often advertise a company's products, but this one had absolutely zero content other than advertising.
I think it said something very meaningful and actionable (admittedly at a high level) about keeping the hot path deterministic: low allocations (including guidelines for GC frequency targets), don't use vthreads (instead use one thread on a pcore that never sleeps, to avoid amdahl), caution about techiques that work at p99 and fail at 99.999... This is not typical for "enterprise" Java, and is an interesting article…
Good feedback
Re: Why low-latency Java still requires discipline?
#37I get that blog posts often advertise a company's products, but this one had absolutely zero content other than advertising.
Yes, disappointing given that Chronicle actually does have legit expertise here. (Chronicle Map is not very well-known even in the Java space but it's by far the best larger-than-memory Map available.)
Thank you for the feedback
Re: Why low-latency Java still requires discipline?
#38How about not using Java? Then you can have low latency. Average go, rust, c++ and c will outperform amazing java programs, and the former will also be way way more easy to run, troubleshoot, interpret logs from. Java is usch garbage in every stack.
> Average go, rust, c++ and c will outperform amazing java programs
Everyone that has a minimal experience with these knows it is not true.
While performance ceiling is likely higher in C/C++/Rust (but not Go), when we move beyond microbenchmarks Java provides competitive performance with much better ergonomics. Not to mention strictly superior tooling.
Re: Why low-latency Java still requires discipline?
#39How about not using Java? Then you can have low latency. Average go, rust, c++ and c will outperform amazing java programs, and the former will also be way way more easy to run, troubleshoot, interpret logs from. Java is usch garbage in every stack.
Java is a great language and runtime. Java being slow is most often a pebcak scenario. There are massive, fast, low latency, Java systems out there that have been written by competent developers.
Re: Why low-latency Java still requires discipline?
#40Earlier quoted context omitted.
I need a UI which runs well on Windows, MacOS, and Linux, without having to build three different ones. Swing is still easily the best, most consistent, and most native-feeling cross-platform environment. It's much better than QT and GTK in most respects. And Java also runs elegantly on a little platform you may know as Android. I have high hopes for go and rust. But until they have mature UIs, they're out (for me).…
In heavily performance-engineered code having a GC coalesce memory is a pessimization in all cases. I've done a lot of performance engineering in both C++ and Java. Every optimization available in Java also exists in C++ but the reverse is not true, which is why C++ is always faster. Every example I have ever seen of Java being faster than C++ was just poorly optimized code. The heuristic I use is that heavily optimi…
What type and size of applications have you worked on?
@pron, who works on the JVM and is a C++ expert, has been writing a lot recently about low level languages becoming increasingly impossible to optimize with more LOC and more people working on it (dozens, or even hundreds of developers).
The idea being be a language with an aggressive JIT, moving garbage collector, and bump allocation is going to allocate/deallocte faster than malloc, and get the most important average optimizations at all times, where as the equivalent low level app will struggle to get the ideal allocation pattern the more people that work on it, or be forced to use slow dynamic dispatch. He wouldn't describe a GC as pessimistic, he'd say it's literally faster, and that's why he (a C++ programmer) works on it. The JVM was built by people saying "we are doing a handful of the same things in C++ all the time."
There are specific cases where it's good to have low level control, but they aren't everywhere.