Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GC Tuning Confessions of a Performance Engineer

slideshare.net

Post reply on HN
Print view View original

61–70 of 98 posts

Re: GC Tuning Confessions of a Performance Engineer

#61

Post by pron »

pcwalton wrote: post #24
Dewie3 wrote: post #16

Earlier quoted context omitted.

Not to mention that you get to hire performance engineers . People may have overestimated their utility (that they are good for less things then commonly thought). Maybe not to the degree that the stereotypical C/++ would believe, but still an overestimation. Maybe we just need to use more abstractions in the middle between manual and fully automatic memory management, like region-based memory management (just an exa…

I think the solution is to use the right memory management tool for the job. pron is right that GC can be helpful for concurrent data structures (where the non-GC approach, hazard pointers, is basically just a GC), for instance. GC is also great when your objects have truly dynamic lifetime, such as open files in Unix from the kernel's point of view. But GC for all data tends to be overkill when your goal is to maxim…

I think we should look at the problem the other way around: it is manual memory management that is overkill because there are exactly four kinds of object memory scope:

* stack scope * transaction scope (for some definition of transaction -- it can be, say, a frame in a game, or a request in a web server) * arbitrary (database or any shared data structure) * permanent

For the stack scope, we have the stack. For the arbitrary scope, a GC is invaluable. For the permanent scope, it doesn't matter too much whether you have a GC or not (yes, vitalyd is going to mention traversals, but if the traversals are interesting, they point to objects in the arbitrary cost anyway). This leaves us the transaction cost. Now, I think it is far easier to manage a stack scope in a GCed environment than an arbitrary scope in a manual environment, and, in fact, some GCs, such as HotSpot's G1 should (yes, vitalyd, it doesn't always work) figure out the tranaction boundary automatically, but even if they don't, it's fairly easy to get that functionality with object pooling. However, to get the absolute best we shouldn't add a GC to a manually managed environment, but add arena collections to a GCed environment, which is exactly what RTSJ (realtime Java) does with scoped memory (making sure there are no references from potentially longer scopes into the arena).

To sum up, for the absolute best performance, a GCed environment + arenas has all you need. There is absolutely no need for a per-object malloc/free, and reference counting is neither here nor there (contention, cycles).

Re: GC Tuning Confessions of a Performance Engineer

#62

Post by pcwalton »

pron wrote: post #61
pcwalton wrote: post #24

Earlier quoted context omitted.

I think the solution is to use the right memory management tool for the job. pron is right that GC can be helpful for concurrent data structures (where the non-GC approach, hazard pointers, is basically just a GC), for instance. GC is also great when your objects have truly dynamic lifetime, such as open files in Unix from the kernel's point of view. But GC for all data tends to be overkill when your goal is to maxim…

I think we should look at the problem the other way around: it is manual memory management that is overkill because there are exactly four kinds of object memory scope: * stack scope * transaction scope (for some definition of transaction -- it can be, say, a frame in a game, or a request in a web server) * arbitrary (database or any shared data structure) * permanent For the stack scope, we have the stack. For the a…

> yes, vitalyd is going to mention traversals, but if the traversals are interesting, they point to objects in the arbitrary cost anyway

I'm going to mention this too. I don't understand what you mean by "they point to objects in the arbitrary cost anyway". The fact of the matter is that you have to trace all objects at some point.

> Now, I think it is far easier to manage a stack scope in a GCed environment than an arbitrary scope in a manual environment, and, in fact, some GCs, such as HotSpot's G1 should (yes, vitalyd, it doesn't always work) figure out the tranaction boundary automatically, but even if they don't, it's fairly easy to get that functionality with object pooling.

Object pooling is just a (limited, error-prone, poorly-performing) form of manual memory management.

> reference counting is neither here nor there (contention, cycles).

Contention isn't a problem if you don't touch the reference counts much. Cycles aren't a problem if you don't have cycles.

Re: GC Tuning Confessions of a Performance Engineer

#63

Post by pron »

phamilton wrote: post #7
pron wrote: post #3

Earlier quoted context omitted.

In 80% of the cases, simple GC ergonomics (i.e. let HotSpot figure out the tuning) are more than ok, and in 95% of the cases, trivial tuning is enough. The thing with Erlang is that the GC doesn't work on any shared memory data structure (like ETS), so you pretty much have to delegate any shared data to an out-of-process database, even in simple cases that are easily addressed with ConcurrentHashMap, ConcurrentSkipLi…

I generally use an Agent in Elixir for anything shared and there's very little cognitive overhead (though perf does take a hit). Agent.cast/2 is helpful for low latency modifications (at the cost of losing back pressure). As stated, it's a tradeoff.

But, if I'm not mistaken, an agent isn't concurrent. You make your shared data a bottleneck which is exactly what concurrent data strctcures are meant to prevent.

Re: GC Tuning Confessions of a Performance Engineer

#64

Post by pron »

jokoon wrote: post #23

I wonder, isn't it working on the stack much faster than working on the heap anyway? So when you query data on the heap, shouldn't you just query larger objects to put it on the stack, and work from there, instead of using the heap so often ? With that in mind wouldn't that render garbage collecting almost irrelevant if your code is well designed, by not working too much on the heap ? It's true that more ram makes th…

Sure, but on modern servers, the amount of data you can put on the stack (or stacks) amounts to no more a few percent of RAM. The smartest thing you can do with the rest, is put as much of your DB data there, and that data has an arbitrary lifetime, and is read -- and written -- concurrently. So most of the memory activity on large servers is going to be on the heap. If you put on the stack everything that can be done on the stack, you've reduced your heap usage by, say, 2%. True, the kind of memory work on the stack and on the heap is very different, but the two problems are of entirely different magnitude and nature.

I think the reason many people say "just use the stack more", is that most people delegate the hard work to a DB, and let the DB engineers worry about using the heap well. But the best way to use a DB is if it is in memory and in process.

Re: GC Tuning Confessions of a Performance Engineer

#65

Post by pron »

vitalyd wrote: post #58
pron wrote: post #42

Earlier quoted context omitted.

While everything you say is 100% true, it is also a second-order effect, with a much lower magnitude than the primary GC performance behavior for short-lived objects. Obviously stack allocation is preferable to TLAB allocation (and there's no reason to allocate objects with stack scope on the heap), if only for the fact that it never triggers a collection. Nevertheless, Java allocation/collection of short-lived objec…

I'm not sure memory locality effects can be considered secondary, unless their effects are completely dwarfed by something else the app is doing (e.g. there's no point in discussing this topic for i/o bound workloads). I don't think malloc/free should enter this conversation because languages that use malloc/free do so very infrequently, and for the cases where dynamic memory needs to be allocated frequently, they us…

> I'm not sure memory locality effects can be considered secondary

What is secondary isn't the TLAB/stack performance ratio, but that ratio vs malloc/TLAB.

Also, I'm not sure why you think locality matters much here in the case of stack reuse. Within each frame, the stores always come first, and those go in the store buffer (and the reads are from the store buffer, too), so those are pretty benign cache misses.

> which, I argue, is rare in properly written applications

Sure, it is rare in "well written applications", but how costly is it to write a well-written application in a large team, and how much extra performance can you get? Remember, we're not talking about a DSP, a microcontroller or a net router when we're discussing GCs, but big, complex applications. Nobody is claiming you can't beat a GC given enough work (though it's harder the more concurrency is involved).

Also, think about what kind of data we're talking about. The interesting data is database data, and that has both arbitrary lifetime as well as concurrent read/write. And if you don't use malloc/free, at best you need to write your own manual memory allocator which is just as complex, and at worst you basically need to write your own GC.

Re: GC Tuning Confessions of a Performance Engineer

#66

Post by dumael »

vitalyd wrote: post #50
dumael wrote: post #40

Earlier quoted context omitted.

I'm almost positive that x86 doesn't really have special stack instructions, aside from call (push pc +sizeof(call)) onto $esp and jump. I know MIPS (and possibly other RISC architectures) don't have an implicit 'stack' register, just one that used by convention. They do have jump and link instructions which write $pc+4 into another register. Afaik, LLVM and quite possibly GCC just add/subtract $esp for the initial a…

There's also push/pop.

You got me there. I do now recall staring at the disassembly of LLVM produced code (LLVM 3.0ish) and seeing pushs/pops.

Re: GC Tuning Confessions of a Performance Engineer

#67

Post by vitalyd »

pcwalton wrote: post #62
pron wrote: post #61

Earlier quoted context omitted.

I think we should look at the problem the other way around: it is manual memory management that is overkill because there are exactly four kinds of object memory scope: * stack scope * transaction scope (for some definition of transaction -- it can be, say, a frame in a game, or a request in a web server) * arbitrary (database or any shared data structure) * permanent For the stack scope, we have the stack. For the a…

> yes, vitalyd is going to mention traversals, but if the traversals are interesting, they point to objects in the arbitrary cost anyway I'm going to mention this too. I don't understand what you mean by "they point to objects in the arbitrary cost anyway". The fact of the matter is that you have to trace all objects at some point. > Now, I think it is far easier to manage a stack scope in a GCed environment than an…

>I'm going to mention this too. I don't understand what you mean by "they point to objects in the arbitrary cost anyway". The fact of the matter is that you have to trace all objects at some point.

I think he means that if you have object references, then those references are interesting for the application itself, and not just for the GC tracer. However, I don't buy this statement simply because, even if you elect to store things as references rather than interior allocs, your app will only pointer chase the things it cares about, whereas the GC tracer may chase pointers that your app wouldn't otherwise.

@Ron:

The "arbitrary" scope better not be really arbitrary since even GCs are typically tuned for the generational hypothesis: your objects better be either short lived or long lived, anything in between is likely to degrade GC performance. In fact, if you like at some of the big data java solutions, once they reach a certain heap size, GC starts killing them, and they end up building their own semi-hybrid solution of moving stuff off-heap, and then managing that memory themselves (using the exact same block/slab/arena allocation techniques being downplayed here!).

For things like databases where object lifetime is in the hands of the user, arena/slab allocators work just fine. The additional advantage here is that you rarely need to destroy just a single object somewhere, typically it's an entire blob of related stuff. Arena destruction is more efficient here than GC because it's it inherently has more context than GC (the engineer wrote it, afterall). Is it more challenging/harder to implement than simply punting to the GC? Probably yes. But if you run afoul of GC's ergonomics and/or need to start tuning the finer points of GC behavior (i.e. see above comment about most of the big data java projects hitting this wall), the time spent there may end up exceeding arena impl costs.

Re: GC Tuning Confessions of a Performance Engineer

#68

Post by banachtarski »

bd_at_rivenhill wrote: post #30
banachtarski wrote: post #5

Having worked with all sorts of GCs in the past, I basically stopped using them altogether some years ago in favor of manual memory management. I have to say, it's been liberating and so much easier to write performant code since I know what memory I need instead of relying on the computer to guess (and giving the computer hints as to how to guess). I have no doubt that GC research and systems is making good forward…

I think we will start to see less use of GCs in mobile going forward, especially now that Apple has deprecated GC in favor of automatic reference counting. The issues caused by the interaction between the memory hierarchy and the fragmentation that is naturally introduced by GC are difficult to measure, but nevertheless an important component of performance.

Reference counting is still not the same as manual memory management, and there is a heavy cost to reference counting as well. Reference counting trades throughput for latency relative to a garbage collected runtime.

Re: GC Tuning Confessions of a Performance Engineer

#69

Post by banachtarski »

pjmlp wrote: post #46
banachtarski wrote: post #44

Earlier quoted context omitted.

Man, the new generation of programmers are going to be in bad shape if the thought of stack unwinding or managing the heap yourself is left to the "highly skilled." Even if I was interviewing someone who only worked in a memory-managed language, I would hope that they'd at least be capable of understanding how to code without it. These should be the fundamentals.

Well I started coding in 1986, and have seen quite a few screw ups. So yeah, small teams it works great. Now scale that to developer teams > 30 on average, with high turnaround and multiple outside partners coming and going on project basis. I have seen what off-shoring does to C and C++ code bases...

How many engineers work on, say, the Unreal Engine. The linux kernel? Windows? Do you think any of that stuff would work even marginally well running on the JVM (assuming the JVM had native driver support)?

I don't quite follow your claims, nor do I put any stock in the tenure of your programming career.

As for the off-shoring point, off-shoring is going to create questionable quality code regardless of the programming language.

Re: GC Tuning Confessions of a Performance Engineer

#70

Post by vitalyd »

pron wrote: post #65
vitalyd wrote: post #58

Earlier quoted context omitted.

I'm not sure memory locality effects can be considered secondary, unless their effects are completely dwarfed by something else the app is doing (e.g. there's no point in discussing this topic for i/o bound workloads). I don't think malloc/free should enter this conversation because languages that use malloc/free do so very infrequently, and for the cases where dynamic memory needs to be allocated frequently, they us…

> I'm not sure memory locality effects can be considered secondary What is secondary isn't the TLAB/stack performance ratio, but that ratio vs malloc/TLAB. Also, I'm not sure why you think locality matters much here in the case of stack reuse. Within each frame, the stores always come first, and those go in the store buffer (and the reads are from the store buffer, too), so those are pretty benign cache misses. > whi…

Ok, we keep talking about malloc -- which malloc impl are you specifically referring to? There are many allocators out there these days, so let's be a bit more concrete. If not specific name, at least the class of allocator. Most of the common ones you'll find support thread-local allocation buffers, for starters.

>Sure, it is rare in "well written applications", but how costly is it to write a well-written application in a large team, and how much extra performance can you get?

>Also, think about what kind of data we're talking about. The interesting data is database data, and that has both arbitrary lifetime as well as concurrent read/write. And if you don't use malloc/free, at best you need to write your own manual memory allocator which is just as complex, and at worst you basically need to write your own GC.

If we're going to talk about databases, then "well-written" better be one of the top concerns, and team size should be irrelevant to that. And the more mechanically sympathetic of a product you're building (and db's are right up there in pretty much all aspects: cpu, i/o, net, mem, etc), the more you need to have control over those resources.

Have you, for example, looked at how postgresql manages memory? sqlite? redis? memcached? nginx? varnish? And, as I mentioned in the other reply, most of the big data java solutions end up rolling their own off-heap mem management infra using the same techniques as you'd use without GC.

Post reply on HN