Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tune Code Before Your Garbage Collector

blog.vanillajava.blog

Post reply on HN
Print view View original

21–30 of 41 posts

Re: Tune Code Before Your Garbage Collector

#21

Post by cogman10 »

actionfromafar wrote: post #19
drob518 wrote: post #17

Earlier quoted context omitted.

I do something similar for Java Time local dates. Financial data in particular has lots of redundant date info and benefits from being memoized. Converting to epoch millis also works.

If I squint, is this a special kind of heap compression?

Not really heap compression or special, it's just reusing a reference to an object already allocated on the heap.

Right now, if I do this

    LocalDate a = LocalDate.of(2020, 1, 1);
    LocalDate b = LocalDate.of(2020, 1, 1);
A and B reference 2 different object allocations on the heap even though they are the same date. a != b.

In Java, that can be pretty expensive even for an object as light as a LocalDate. By running the cache and doing

    var cache = new HashMap();
    LocalDate a = cache.computeIfAbsent(LocalDate.of(2020, 1, 1), (i)->i);
    LocalDate b = cache.computeIfAbsent(LocalDate.of(2020, 1, 1), (i)->i);
Now you have the situation where `a == b` and you immediately end up dropping the object allocation for b on the next GC.

The technique works best when you have a lot of repeated objects which are immutable. It is also only really needed because Valhalla isn't here. Once "value types" become a thing, then the representation for `LocalDate` inside the JVM can become just the fields and not a reference. The JVM is also free to do the sort of de-duplication optimization all on it's own for larger objects.

Re: Tune Code Before Your Garbage Collector

#22

Post by hyperpape »

cogman10 wrote: post #12
layer8 wrote: post #10

Earlier quoted context omitted.

All common logging backends create a LogEvent or similar object for each logging call, and logging calls also typically construct new strings, which usually means a new StringBuilder object, its internal array (multiple ones if it grows), the final array it is copied to, and the String object that wraps that array. These are typically short-lived objects and therefore cheap. Nevertheless, continually creating many su…

Cheap, not free, and even pretty simple to accidentally fool the GC on the lifetime of these objects. Consider, for example, if you have a log message like this logger.info("Hello {}", myOldObject); if "myOldObject" is large enough or contains references to large things or has just been around for a while, it may be a part of OldGen at this point. And if that's the case, the LogEvent objects will end up automatically…

> And if that's the case, the LogEvent objects will end up automatically promoted to OldGen.

Why do you think this would happen? There's no mechanism that makes young gen objects that reference old gen objects (or are referenced by old gen objects) get promoted faster. You have to survive a certain number of collections.

Re: Tune Code Before Your Garbage Collector

#23

Post by usrnm »

bigfishrunning wrote: post #7
einpoklum wrote: post #5

Garbage collector? To quote Bjarne Stroustrup: > I don't like garbage. I don't like littering. My ideal is to eliminate the need for a garbage collector by not producing any garbage. That is now possible.

[flagged]

You're wrote those words using technology built in C++. Just saying

Re: Tune Code Before Your Garbage Collector

#24

Post by cogman10 »

hyperpape wrote: post #22
cogman10 wrote: post #12

Earlier quoted context omitted.

Cheap, not free, and even pretty simple to accidentally fool the GC on the lifetime of these objects. Consider, for example, if you have a log message like this logger.info("Hello {}", myOldObject); if "myOldObject" is large enough or contains references to large things or has just been around for a while, it may be a part of OldGen at this point. And if that's the case, the LogEvent objects will end up automatically…

> And if that's the case, the LogEvent objects will end up automatically promoted to OldGen. Why do you think this would happen? There's no mechanism that makes young gen objects that reference old gen objects (or are referenced by old gen objects) get promoted faster. You have to survive a certain number of collections.

Ah shoot, you're right this wouldn't be what I'm thinking.

I was thinking of "GC Nepotism" [1]. That's the case where an object in old gen pointing to an object in new gen will automatically promote that new gen object into old gen. This can be particularly problematic with graph structures.

[1] https://psy-lob-saw.blogspot.com/2016/03/gc-nepotism-and-lin...

Re: Tune Code Before Your Garbage Collector

#25

Post by hinkley »

motoboi wrote: post #8
senderista wrote: post #4

Earlier quoted context omitted.

I worked on Java code at AWS for a few years and nobody tried to optimize allocations. Then I changed jobs and started working on a Java MPP database and my first code review was brutal. You were expected to avoid allocations as much as possible (mostly by using the SoA pattern everywhere). At that scale no GC could save you from excessive allocations.

I believe the whole string vs stringbuffer that later was made redundant by compiler contributed to that vision. People started dismissing allocation discipline as a thing from the past because "that thing was solved a lot ago and the compiler now is smart enough". Well, for string, yes, but not for arbitrary objects.

I worked on projects where we indeed needed the manual stringbuffer refactor on hot paths in the code. It's not difficult work but boy is it tedious. Spent a lot of time with headphones on during those days. Only to work on another project a few years later where I paid for my sins by reversing the same change on someone else's codebase.

There's a version of Java, I can't recall which but I want to say 4? Maybe 3? Where someone rewrote parts of the Swing backend as native code to speed it up. Then Hotspot got good enough in the next version that the generated code was faster than the native code + FFI overhead. FFI was pretty high at the time. So they reverted the native code migration and went back to the old code.

Re: Tune Code Before Your Garbage Collector

#26

Post by hinkley »

cogman10 wrote: post #9
motoboi wrote: post #8

Earlier quoted context omitted.

I believe the whole string vs stringbuffer that later was made redundant by compiler contributed to that vision. People started dismissing allocation discipline as a thing from the past because "that thing was solved a lot ago and the compiler now is smart enough". Well, for string, yes, but not for arbitrary objects.

The most surprising allocation pressure I constantly run into is primitive boxing. The JVM does heroics to try and avoid it as much as possible, but when you end up with some primitive boxing in a hotspot the amount of GC pressure that creates can be unreal.

The especially nasty thing about that boxing pressure is that when I still paid attention to the JVM, they had some sort of cache for boxing of numbers under about 128, which meant that if you did tests with toy inputs, identity checks would pass in code that needed an equality check instead, and benchmarks built the same way would see no GC pressure in those call trees but see quite a lot of pressure in production. I remember being really mad, almost to the point of a sense of betrayal, when I learned that. It's kind of a DieselGate but with more plausible deniability. You can't say it was deliberate, but if you already had trust concerns with that person it would move the dial farther into the red.

Re: Tune Code Before Your Garbage Collector

#27

Post by hinkley »

drob518 wrote: post #17
cogman10 wrote: post #15

Earlier quoted context omitted.

One of my least proud (most proud?) hacks when working with very large data sets is something like this Map intCache = new HashMap (); while (loading) { Integer feild1 = intCache.computeIfAbsent(getField1(), (i)->i); } This is a terrible thing that shouldn't be as useful as it is to us... but it is really useful. We have a bunch of objects that can optionally have Integer values (hence a null is valid) but those int…

I do something similar for Java Time local dates. Financial data in particular has lots of redundant date info and benefits from being memoized. Converting to epoch millis also works.

There was a generation of date processing in Java when the class wasn't reentrant. The API was functionally flavored though, so people would instantiate one instance configured how they liked and reuse it across all calls. You would only see this problem under heavy load, such as in production, and it basically took encountering someone for whom it had already happened to spot what was going on. I think I found out from the Java issue tracker, thanks to some other dev filing an accurate bug report, but then I was that person for others at a full handful of other jobs afterward. Everyone was surprised, as one would be. They eventually fixed it but that was busted for a long long time.

Re: Tune Code Before Your Garbage Collector

#28

Post by to11mtm »

cogman10 wrote: post #15
drob518 wrote: post #13

Earlier quoted context omitted.

Yep, and sometimes just a small code change can flip on boxing.

One of my least proud (most proud?) hacks when working with very large data sets is something like this Map intCache = new HashMap (); while (loading) { Integer feild1 = intCache.computeIfAbsent(getField1(), (i)->i); } This is a terrible thing that shouldn't be as useful as it is to us... but it is really useful. We have a bunch of objects that can optionally have Integer values (hence a null is valid) but those int…

I mean FWIW this looks to me much like the sort of hacks I had to get into last time I worked with Java so I can't really blame you....

> Valhalla can't come soon enough for us.

This will be interesting to see for sure, I think it will raise the bar for competitors as well; .NET GC has lingered in some ways for a while in progressing [0][1], there is a long standing github discussion around a lower latency GC where there is a potential alternative [2] but nothing really signaling that it could be integrated in future as an option. Valhalla might finally put enough pressure on microsoft to do something about the lag in this space.

[0] - The inferred stackalloc stuff on the JIT level is awesome but I don't count that as improvement to actual GC

[1] - Pinned allocs took a long time and we still can't get aligned pinned allocs (have to manually pad instead)

[2] - https://github.com/dotnet/runtime/discussions/115627

Re: Tune Code Before Your Garbage Collector

#29

Post by einpoklum »

scj wrote: post #20
drob518 wrote: post #14

Earlier quoted context omitted.

I prefer to see C++ as a failed experiment that just keeps going and going rather than garbage. The software industry learned a lot from it, both good and bad. But yea, I haven’t programmed in it since the late 1990s.

I'd phrase it differently, C++ was a set of power-to-performance trade-offs that were optimal in the 1990s. Time has moved on. More importantly, a typical 1990s C++ dev was likely someone who learned assembly, then C or C++. Meaning they already knew how to control hardware / memory allocation, and C++ was just a new set of abstraction tools. It was a step forward for them. To modern devs, C++ is a step backwards. An…

Actually, C++ was rather poor in the 1990s if you ask me (albeit still very usable). Time has moved on - but so has the language. Its implementation tradeoffs were much better IMNSHO after 2011; but it wasn't there yet. And it still isn't! It has a lot of warts that have to stay for backwards compatibility (which is a design goal); and then, it has annoyances I can't believe are not yet addressed (like - where is my 'restrict' keyword, damn it?!)

Anyway, your view of the 1990s devs is incorrect. Almost no programmers who took up C++ learned assembly first (and few ever learned assembly). I believe most of them learned Pascal, or C, or scripting languages like Perl or Tcl or Unix shell scripts. Some may have learned Lisp or some ML variant as their first language, or Fortran 90. They didn't take a step back, they switched to a different set of language design goals and tradeoffs, and found it, well, serviceable.

It is indeed a bit peculiar that the language has had this much staying power. For C, it's much more understandable - because C is such a small and simple language (and one which, as you suggested, often feels like a bunch of syntactic sugar over PDP-7 assembly). But C++ is big, and has its baggage and warts and flaws. I think it's probably because it's been able to adapt and stretch just enough under the influence of trends in programming languages, for people not to ditch it for something new. Maybe Rust will change that; but - C++ might very well "eat its lunch".

Post reply on HN