Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dissecting the CPU-memory relationship in garbage collection (OpenJDK 26)

norlinder.nu

Post reply on HN
Print view View original

31–38 of 38 posts

Re: Dissecting the CPU-memory relationship in garbage collection (OpenJDK 26)

#31

Post by mattclarkdotnet »

noelwelsh wrote: post #23
mattclarkdotnet wrote: post #12

Sorry if this is obvious to Java experts, but much as parallel GC is fine for batch workloads, is there a case for explicit GC control for web workloads? For example a single request to a web server will create a bunch of objects, but then when it completes 200ms later they can all be destroyed, so why even run GC during the request thread execution?

There are a few ways of looking at this: - Purely on the JVM, you probably want ZGC (or Shenandoah) because latency is more important than throughput. - On Erlang / the BEAM VM, each thread gets its own private heap, so GC is a per thread operation. If the request doesn't spill over the heap then GC would never need to run during a request handler and all memory could be reclaimed when the handler finishes. - There c…

Thank you, that’s what I came here to learn!

Re: Dissecting the CPU-memory relationship in garbage collection (OpenJDK 26)

#32

Post by torginus »

I think a very serious issue with GC is that:

- The number of edges in a graph tend to scale superlinearly with heap size, as the number of edges possible in a graph are quadratic wrt no of objects.

- Memory bandwidth hasn't been scaling very much during the past decade and a half, even compared to memory size. It's also not a thing people think about or even easy to display in any performance monitoring tool.

But considering if you had a machine 15 years ago with 4GB or ram that could be read at 15GB/s, and now you have one with 32GB that can be read at 60GB/s, it means that your bandwidth compared to heap size has halved. Considering the quadratic nature of references, the 'amplification factor', the number of times you have to revisit an already visited block of memory is higher as well.

This is in addition to the cache trashing issues mentioned in the post.

If you need to read the whole heap, this sets a lower bound on how much time the GC will take ~0.25s on the old machine, ~0.5s on the new one.

Suppose your GC triggers a memory bandwidth issue - how do you even profile for that? This is kind of an invisible resource that just gets used up.

Re: Dissecting the CPU-memory relationship in garbage collection (OpenJDK 26)

#33

Post by magicalhippo »

jonasn wrote: post #7
firefly2000 wrote: post #5

Are there plans to elucidate implicit GC costs as well?

Great question! I actually just touched on this in another thread that went up right around the same time you asked this. It is clearly the next big frontier! The short answer is: It's something I'm actively thinking about, but instrumenting micro-level events (like ZGC's load barriers or G1's write barriers) directly inside application threads without destroying throughput (or creating observer effects invalidating…

> instrumenting micro-level events (like ZGC's load barriers or G1's write barriers) directly inside application threads without destroying throughput (or creating observer effects invalidating the measurements) is incredibly difficult

I've used a sampling profiler with success to find lock contention in heavily multithreaded code, but I guess there are some details that makes it not viable for this?

Re: Dissecting the CPU-memory relationship in garbage collection (OpenJDK 26)

#34

Post by hinkley »

noelwelsh wrote: post #23
mattclarkdotnet wrote: post #12

Sorry if this is obvious to Java experts, but much as parallel GC is fine for batch workloads, is there a case for explicit GC control for web workloads? For example a single request to a web server will create a bunch of objects, but then when it completes 200ms later they can all be destroyed, so why even run GC during the request thread execution?

There are a few ways of looking at this: - Purely on the JVM, you probably want ZGC (or Shenandoah) because latency is more important than throughput. - On Erlang / the BEAM VM, each thread gets its own private heap, so GC is a per thread operation. If the request doesn't spill over the heap then GC would never need to run during a request handler and all memory could be reclaimed when the handler finishes. - There c…

See also arena allocation, for realtime systems. But those systems typically require that any task have a reasonably tight upper bound on memory usage.

Re: Dissecting the CPU-memory relationship in garbage collection (OpenJDK 26)

#35

Post by hinkley »

mattclarkdotnet wrote: post #19
jacobn wrote: post #13

Earlier quoted context omitted.

Most web request cases where you care about performance probably have multiple parallel web requests, so there’s no clean separation possible?

Sure, but each request has its own context. Shared resources like DB connection pools will be longer lived but by definition they aren’t alllcated by the request thread. So why not simply exempt everything allocated by a request thread from GC, and simply destroy it on request completion?

Generational GC assumes that short lived objects tend to come in groups, which is probably the best you can do in an OO language with shared everything.

Re: Dissecting the CPU-memory relationship in garbage collection (OpenJDK 26)

#36

Post by yvdriess »

spockz wrote: post #29
jonasn wrote: post #4

Earlier quoted context omitted.

Thanks for reading! Your current method, pushing the load until the GC spirals and then comparing the CPU utilization, is exactly the painful, trial-and-error approach I'm hoping this new API helps alleviate. You've hit on the exact next frontier of GC observability. The API in JDK 26 tracks the explicit GC cost (the work done by the actual GC threads). Tracking the implicit costs, like the overhead of ZGC's load bar…

After writing my previous post I was wondering, do we actually need to instrument the barrier events and other code tied to a GC? Currently we benchmark our application with different GC at different settings and resource constraints and the we pick one sizing and settings combination that we like (read most work/totalcpu that is still fits within the allocation constraints of our clusters). What ultimately matters f…

The problem is that there is no baseline for measuring GC overhead. You cannot turn it off, you can only replace and compare with different strategies. For example sbrk is technically a noop GC, but that also has overhead and impact because it will not compact objects and give you bad cache behavior. (It illustrates the OP's point that it is not enough to measure pauses, sbrk has no pauses but gets outperformed easily.)

You could stop collecting performance counters around GC phases, but you even if you are not measuring the CPU still runs through its instructions, causing the second order effects. And as you mentioned too-short-to-measure barriers and other bookkeeping overheads (updating ref counters etc) or simply the fact that some tag bits or object slots are reserved all impact performance.

There is a good write-up of the problem and a way to estimate the cost based on different GC strategies, as you suggested, here: https://arxiv.org/abs/2112.07880

The way I found to measure a no-GC baseline is to compare them in an accurate workload performance simulator. Mark all GC and allocator related code regions and have the simulator skip all those instructions. Critically that needs to be a simulator that does not deal with the functional simulation, but gets it's instructions from a functional simulator, emulator or PIN tool that does execute everything. It's laborious, not very fast and impractical for production work. But, it's the only way I found to answer a question like "What is the absolite overhead of memory management in Python?". (Answer: lower bound walltime sits around +25% avg, heavily depending on the pyperformance benchmark)

Re: Dissecting the CPU-memory relationship in garbage collection (OpenJDK 26)

#37

Post by abbeyj »

jonasn wrote: post #2

Hi HN, I'm the author of this post and a JVM engineer working on OpenJDK. I've spent the last few years researching GC for my PhD and realized that the ecosystem lacked standard tools to quantify GC CPU overhead—especially with modern concurrent collectors where pause times don't tell the whole story. To fix this blind spot, I built a new telemetry framework into OpenJDK 26. This post walks through the CPU-memory tra…

I'm a bit confused about the colors used in the CPU graphs. In the first graphs it looks like green means that the application is running and red means that the GC is running. But once we get to Figure 4 then red means the GC is running (on the GC threads) or nothing is running (on the Main thread)? If red always means that GC work is being done on that thread then this is inconsistent with the text that says "By distributing reclamation work across both cores..." since we would have three threads running at once. Once you move to the concurrent GC figures you definitely have three things running at once. Unless you're assuming SMT with each core running two threads?

In Figure 3 you somehow have 101% wall time. :)

Re: Dissecting the CPU-memory relationship in garbage collection (OpenJDK 26)

#38

Post by jonasn »

abbeyj wrote: post #37
jonasn wrote: post #2

Hi HN, I'm the author of this post and a JVM engineer working on OpenJDK. I've spent the last few years researching GC for my PhD and realized that the ecosystem lacked standard tools to quantify GC CPU overhead—especially with modern concurrent collectors where pause times don't tell the whole story. To fix this blind spot, I built a new telemetry framework into OpenJDK 26. This post walks through the CPU-memory tra…

I'm a bit confused about the colors used in the CPU graphs. In the first graphs it looks like green means that the application is running and red means that the GC is running. But once we get to Figure 4 then red means the GC is running (on the GC threads) or nothing is running (on the Main thread)? If red always means that GC work is being done on that thread then this is inconsistent with the text that says "By dis…

Thanks for the detailed read and the great questions!

Regarding the colors and thread counts in Figure 4: the key piece of context here is that the application thread (the Main thread) is completely paused during this phase. It isn't actually running anything at all. Because the application is halted, only the GC threads are doing active work. Therefore, rather than three threads running at once, we strictly have two things running concurrently. This is a helpful piece of feedback and I'll make sure to make this clearer in future writings.

Good eye on the 101% wall time. That was due to a minor bug in my plotting script that specifically affected the GC plots with no concurrent time. I have corrected this and updated the post. The fixed plot should be visible on the site in a future near you just as soon as the edge caches invalidate.

Post reply on HN