Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dissecting the CPU-memory relationship in garbage collection (OpenJDK 26)

norlinder.nu

Post reply on HN
Print view View original

21–30 of 38 posts

Re: Dissecting the CPU-memory relationship in garbage collection (OpenJDK 26)

#21

Post by NovaX »

mattclarkdotnet wrote: post #19
jacobn wrote: post #13

Earlier quoted context omitted.

Most web request cases where you care about performance probably have multiple parallel web requests, so there’s no clean separation possible?

Sure, but each request has its own context. Shared resources like DB connection pools will be longer lived but by definition they aren’t alllcated by the request thread. So why not simply exempt everything allocated by a request thread from GC, and simply destroy it on request completion?

Go tried that [1], a failed experiment that was a complex NIH version of the generational hypothesis. They currently use a CMS-stye collector.

[1] https://docs.google.com/document/d/1gCsFxXamW8RRvOe5hECz98Ft...

Re: Dissecting the CPU-memory relationship in garbage collection (OpenJDK 26)

#22

Post by zokier »

yunnpp wrote: post #15
jacobn wrote: post #13

Earlier quoted context omitted.

Most web request cases where you care about performance probably have multiple parallel web requests, so there’s no clean separation possible?

His question is still valid for latency. That parallel GC in Java still seems to pause threads from a quick search. https://inside.java/2022/08/01/sip062/

That's why we got ZGC and Shenandoah, and their generational variants, which have very low pause times (in the order of 1 ms)

Re: Dissecting the CPU-memory relationship in garbage collection (OpenJDK 26)

#23

Post by noelwelsh »

mattclarkdotnet wrote: post #12

Sorry if this is obvious to Java experts, but much as parallel GC is fine for batch workloads, is there a case for explicit GC control for web workloads? For example a single request to a web server will create a bunch of objects, but then when it completes 200ms later they can all be destroyed, so why even run GC during the request thread execution?

There are a few ways of looking at this:

- Purely on the JVM, you probably want ZGC (or Shenandoah) because latency is more important than throughput.

- On Erlang / the BEAM VM, each thread gets its own private heap, so GC is a per thread operation. If the request doesn't spill over the heap then GC would never need to run during a request handler and all memory could be reclaimed when the handler finishes.

- There can still be cases where a request handler allocates memory that is no solely owned by it. E.g. if it causes a new database connection to be allocated in a connection pool, that connection is not owned by the request handler and should not be deallocated when the handler finishes.

- The general idea you're getting at is often called "memory regions": you can point to a scope in the code and say "all the memory can be freed when this scope exits". In this case the scope is the request handler. It's the same idea behind arena or slab memory allocation. There are languages that can encode this, and do safe automatic memory management without GC. Rust is an obvious example, but I don't find it very ergonomic. I think the OxCaml [1] and Scala 3 [2] approaches are better.

[1]: https://oxcaml.org/documentation/stack-allocation/reference/

[2]: https://docs.scala-lang.org/scala3/reference/experimental/cc...

Re: Dissecting the CPU-memory relationship in garbage collection (OpenJDK 26)

#24

Post by jonasn »

exabrial wrote: post #16
jonasn wrote: post #2

Hi HN, I'm the author of this post and a JVM engineer working on OpenJDK. I've spent the last few years researching GC for my PhD and realized that the ecosystem lacked standard tools to quantify GC CPU overhead—especially with modern concurrent collectors where pause times don't tell the whole story. To fix this blind spot, I built a new telemetry framework into OpenJDK 26. This post walks through the CPU-memory tra…

I just want to say this is an incredibly detailed, well written, and beautifully illustrated article. Solid work.

Thanks! I really appreciate that. I spent a lot of time trying to nail the illustrations so I'm really glad it landed well. :-)

Re: Dissecting the CPU-memory relationship in garbage collection (OpenJDK 26)

#25

Post by jonasn »

sitta wrote: post #18
jonasn wrote: post #2

Hi HN, I'm the author of this post and a JVM engineer working on OpenJDK. I've spent the last few years researching GC for my PhD and realized that the ecosystem lacked standard tools to quantify GC CPU overhead—especially with modern concurrent collectors where pause times don't tell the whole story. To fix this blind spot, I built a new telemetry framework into OpenJDK 26. This post walks through the CPU-memory tra…

Great article! Will the new metric be exposed in JFR recordings as well?

Thanks!

It is not currently exposed in JFR for JDK 26, but I agree that it would be the logical next step. Now that the underlying telemetry framework (cpuTimeUsage.hpp) is in place within HotSpot, wiring it up to JFR events would be a natural extension.

Re: Dissecting the CPU-memory relationship in garbage collection (OpenJDK 26)

#26

Post by jonasn »

firefly2000 wrote: post #11
jonasn wrote: post #7

Earlier quoted context omitted.

Great question! I actually just touched on this in another thread that went up right around the same time you asked this. It is clearly the next big frontier! The short answer is: It's something I'm actively thinking about, but instrumenting micro-level events (like ZGC's load barriers or G1's write barriers) directly inside application threads without destroying throughput (or creating observer effects invalidating…

Do you think it can be done by adjusting GC aggressiveness (or even disabling it for short periods of time) and correlating it with execution time?

That is spot on. Effectively disabling GC to establish a baseline is exactly the methodology used in the Blackburn & Hosking paper [1] I referenced.

In general, for a production JVM like HotSpot, the implicit cost comes largely from the barriers (instructions baked directly into the application code). So even if we disable GC cycles, those barriers are still executing.

If we were to remove barriers during execution, maintaining correctness becomes the bottleneck. We would need a way to ensure we don't mark a live (reachable) object as dead the moment we re-enable the collector.

[1] https://dl.acm.org/doi/pdf/10.1145/1029873.1029891

Re: Dissecting the CPU-memory relationship in garbage collection (OpenJDK 26)

#27

Post by babol »

jonasn wrote: post #26
firefly2000 wrote: post #11

Earlier quoted context omitted.

Do you think it can be done by adjusting GC aggressiveness (or even disabling it for short periods of time) and correlating it with execution time?

That is spot on. Effectively disabling GC to establish a baseline is exactly the methodology used in the Blackburn & Hosking paper [1] I referenced. In general, for a production JVM like HotSpot, the implicit cost comes largely from the barriers (instructions baked directly into the application code). So even if we disable GC cycles, those barriers are still executing. If we were to remove barriers during execution,…

Would running an application with chosen GC, subtracting GC time reported by methods You introduced, and then comparing with Epsilong-based run be a good estimate of barrier overhead ?

Thank you for the well written article!

Re: Dissecting the CPU-memory relationship in garbage collection (OpenJDK 26)

#28

Post by jonasn »

babol wrote: post #27
jonasn wrote: post #26

Earlier quoted context omitted.

That is spot on. Effectively disabling GC to establish a baseline is exactly the methodology used in the Blackburn & Hosking paper [1] I referenced. In general, for a production JVM like HotSpot, the implicit cost comes largely from the barriers (instructions baked directly into the application code). So even if we disable GC cycles, those barriers are still executing. If we were to remove barriers during execution,…

Would running an application with chosen GC, subtracting GC time reported by methods You introduced, and then comparing with Epsilong-based run be a good estimate of barrier overhead ? Thank you for the well written article!

That is a creative idea, but unfortunately, Epsilon changes the execution profile too much to act as a clean baseline for barrier costs.

One huge issue is spatial locality. Epsilon never reclaims, whereas other GCs reclaim and reuse memory blocks. This means their L2/L3 cache hit rates will be fundamentally different.

If you compare them, the delta wouldn't just be the barrier overhead; it would be the barrier overhead mixed with completely different CPU cache behaviors, memory layout etc. The GC is a complex feedback loop, so results from Epsilon are rarely directly transferable to a "real" system.

Re: Dissecting the CPU-memory relationship in garbage collection (OpenJDK 26)

#29

Post by spockz »

jonasn wrote: post #4
spockz wrote: post #3

Earlier quoted context omitted.

Thank you for this interface! It will definitely help in tracking down GC related performance issues or in selecting optimal settings. One thing that I still struggle with, is to see how much penalty our application threads suffer from other work, say GC. In the blog you mention that GC is not only impacting by cpu doing work like traversing and moving (old/live) objects but also the cost of thread pauses and other b…

Thanks for reading! Your current method, pushing the load until the GC spirals and then comparing the CPU utilization, is exactly the painful, trial-and-error approach I'm hoping this new API helps alleviate. You've hit on the exact next frontier of GC observability. The API in JDK 26 tracks the explicit GC cost (the work done by the actual GC threads). Tracking the implicit costs, like the overhead of ZGC's load bar…

After writing my previous post I was wondering, do we actually need to instrument the barrier events and other code tied to a GC? Currently we benchmark our application with different GC at different settings and resource constraints and the we pick one sizing and settings combination that we like (read most work/totalcpu that is still fits within the allocation constraints of our clusters). What ultimately matters for production is how the app behaves in production.

This will not help directly when developing new (versions) or GC. On the other hand, if we can have a noop GC including omitting any of the barriers etc required for GC to function we can create a baseline for apps. Provided we have enough total memory to run the benchmark in.

Edit: I guess we can then also use perf to compare cache misses between runs with different GC implementations and settings. Not sure how this works out in real life as it will be very CPU, kernel, and other loads dependent.

Re: Dissecting the CPU-memory relationship in garbage collection (OpenJDK 26)

#30

Post by spockz »

cogman10 wrote: post #10
jonasn wrote: post #9

Earlier quoted context omitted.

Every GC algorithm in HotSpot is designed with a specific set of trade-offs in mind. ZGC and G1 are fantastic engineering achievements for applications that require low latency and high responsiveness. However, if you are running a pure batch data pipeline where pause times simply don't matter, Parallel GC remains an incredibly powerful tool and probably the one I would pick for that scenario. By accepting the pauses…

Gotta be honest, I have a hard time arguing for G1 over ZGC. It seems to me like any situation you'd want G1 you probably want ZGC instead. That default 200ms target latency is already pretty long. If you've made that tradeoff for G1 because you wanted lower latency, you probably are going to be happier with ZGC. I also find that the parallel collector is often better than G1, particularly for small heaps. With moder…

We have many apps that run on <1 core just fine for the business logic and run on K8S. If we then use a parallel or concurrent garbage collector it will eat through the cpu limit of the app in a blink causing the process not to be scheduled for several ticks. This introduces more latency than the GC cycles themselves would when using a serial GC than runs frequently enough.
Post reply on HN