JDK 27 G1/Parallel/Serial GC Changes

Generational moving collectors are a very powerful memory management optimisation, but because they necessarily require some "interesting" FFI layer between the ordinary heap and any passing of pointers between the program and the hardware/OS - the very thing low-level languages are designed not to have - this is a powerful, general optimisation (not perfect, but extremely useful in a wide class of programs) that is not available to low-level programming languages. And it's not the only one, BTW. JIT compilers are also designed for "global" average-case optimisations at the cost of precise low-level control over the worst case, which is another thing that low-level languages trade away.

In the most simplistic way, I would say that the precise control that low-level languages are all about helps their performance when programs are small (and can be manually optimised globally) and hurts their performance as programs get large. They have to give up on some optimisations that come at the cost of ceding low-level control, and that includes moving collectors.

> When I say tradeoffs", I mean exactly things like "inherent increased footprint", or my earlier "wasting RAM" point.

Well, that is a real tradeoff, but there's a reason why it's a very attractive one for a huge class of applications. There are two ways of looking at this, which amount to the same thing:

1. Because both RAM and CPU are needed for computation, what matters isn't each of their utilisation values separately but only the more constraining or impactful of the two.

2. Because CPU is needed to use RAM, every CPU cycle you spend effectively takes away some other program's ability to use RAM.

This means that for any amount of CPU utilisation, there is some amount of RAM that is effectively free (i.e. has no additional impact), and the more CPU a program consumes, the more RAM it can consume without it making additional impact. It's easiest to see in the extreme case of a program using 100% CPU: no other program can make progress, and so it doesn't matter how much of the available RAM your program is using - it effectively captures all of it whether it uses it or not. But this scales to any amount of CPU utilisation (not quite linearly). What wastes RAM is not using that "free RAM" to reduce the dominant resource, CPU. And what further determines the RAM/CPU "exchange rate" is the RAM/CPU ratio offered by the hardware, which is more RAM-heavy than some appreciate (it is very hard to find a metal or virtual deployment with less than 1 GB of RAM per core these days - taking into account partial cores in virtual machines - outside of embedded devices).

This means that if you have a memory management algorithm that uses more RAM to help reduce CPU as CPU utilisation rises, that's usually a good thing. And moving collectors work exactly like that. The heap overhead in a generational moving collector is only a function of the allocation rate, and a high allocation rate also means high CPU usage.

My colleague, the main developer of ZGC these days, gave a keynote about this very subject at ISMM: https://youtu.be/mLNFVNXbw7I

> I view them as just years of continued tuning of tradeoffs (heap size vs CPU cycles vs memory cycles), while you them as major breakthrough that makes garbage collection much more desirable. Is that fair?

I say that for many years, the main practical, most "felt" tradeoff of moving collectors has been their STW pauses. With pauses eliminated, there is a qualitative change in the attractiveness of moving collectors, making them more appropriate than other memory management techniques for an even broader class of applications than before. Previously, applications that were very sensitive to tail latencies didn't want moving collectors; now, the tail latency is no longer an issue (unless your application's tail latency tolerance is such that a realtime OS is needed). In other words, I'm saying that moving collectors' most impactful tradeoff is now gone.

It won't, because that is exactly the thing good moving GCs detect and size the young-gen accordingly.

> Using RAM to save CPU is exactly the wrong tradeoff when such a system becomes CPU bound.

Did you mean to write something else, because it's pretty obvious that it's the right tradeoff? If something is CPU-bound, you want to reduce the CPU usage.

> Unless each collection takes less work as the heap increases in size, backing off the GC rate to free CPU instantly drives the system into metastable failure, where the GC becomes more expensive because the GC is expensive.

The whole point of moving collectors is that each collection takes the same amount of work, but you need to do it less frequently as the heap rises. So yes, as they heap grows, moving GCs are supposed to work less. The heap grows as a function of the allocation rate while the CPU devoted to memory management remains the same. That's precisely the optimisation that moving collectors bring.

> Instead of reasoning about this all the time

The whole point is that the GC is what "reasons" about this for you.

> it’s much easier (for me, granted, I am not a typical java developer) to just jam the CPU intensive work on a low priority event queue so that it uses 100% CPU but never blocks low latency stuff, or things about to retire requests.

That's orthogonal. You can do that at least as easily in Java.

> This ends up being easier to deal with than java, since everything is thread safe, allocations are predictable, and there are CPU escape hatches I can use.

Thread safety is orthogonal, and now with ZGC, memory management in Java is more predictable than malloc/free allocators.

> C++ lets me use smart pointers that have exactly the semantics I want, and that are memory safe but racy in practice. Rust makes them actually memory and thread safe, but sometimes adds useless copies, initializations and thread synchronization (or requires unsafe).

Yes, and it's also less efficient and less predictable in the memory management work as programs grow larger.