Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GC Tuning Confessions of a Performance Engineer

slideshare.net

Post reply on HN
Print view View original

51–60 of 98 posts

Re: GC Tuning Confessions of a Performance Engineer

#51

Post by vitalyd »

evanpw wrote: post #33
vitalyd wrote: post #29

Earlier quoted context omitted.

The inherent benefit to the stack is the memory region stays hot in cache due to natural use of the stack. That's your fast reusable buffer for temporaries. In addition, of course, cleanup/reclaim of the stack space is pointer bump, so you get pointer bump allocation and deallocation, effectively.

True, but you could do the same thing in any region of memory. The only thing that using "the stack" buys you is that there are a few special instructions to allocate / deallocate one machine word at a time (if you store the "top-of-heap" pointer in a specific register), and you get a bit more locality by virtue of return addresses being stored next to your locals and temporaries. (For x86 processors at least; maybe…

> True, but you could do the same thing in any region of memory. The only thing that using "the stack" buys you is that there are a few special instructions to allocate / deallocate one machine word at a time (if you store the "top-of-heap" pointer in a specific register)

This is pretty much what a TLAB is in Hotspot JVM, except of course the pointer never moves backwards (once the TLAB is filled up, it retires, and then can get assigned to a different thread to start allocating from the beginning). Each allocation into the TLAB moves allocations further into the region -- there's never any reuse until the TLAB starts afresh.

The stack locality comes from the rest of the stack execution mechanics keeping this region very warm, and the majority of the read/write action to the stack is localized (apart from large stack allocations that may temporarily expand the region's use).

Re: GC Tuning Confessions of a Performance Engineer

#53

Post by TheLoneWolfling »

banachtarski wrote: post #43
duaneb wrote: post #37

Earlier quoted context omitted.

Manual memory management (i.e. not GC) is a little disingenuous because it implies that it's entirely manual. In fact, there are many patterns (e.g. RAII) which make "brain dead" manual memory management possible. I'm not necessarily arguing this is a great way to go for a new team, but there are manual memory management strategies that can definitely scale. Among other things, it's not the only type of resource that…

Agreed. Sometimes, you don't even need RAII. You just let the stack unwind :) But yes, RAII can make it pretty braindead (which is good)

Or arena allocation - which, by the way, is something I wish was more supported in mainstream languages

Re: GC Tuning Confessions of a Performance Engineer

#54

Post by TheLoneWolfling »

astral303 wrote: post #41
jimrandomh wrote: post #34

Performance is nice, but correctness is more important. Complex programs written without any GC tend to have use-after-free and double-free vulnerabilities. Programs written using automatic reference counting only (without a way to catch cycles) tend to have memory leaks. If GC is killing performance, then that is a failure of the specific GC or programming language design. Don't blame the applications programmers fo…

Specifically about ArrayList -- there are specifically a bunch of projects (Goldman Sachs Collections, trove, FastUtil, Koloboke) that solve the array of primitives problem. I would say that's a Java-standard-libary-specific problem.

And then you have to do a copy just to pass it around to any other external libraries. And you lose the advantages of generics.

You end up with 9 copies of everything, that are 99% the same except for a couple find-replaces. If not more. (For instance, if you have a method that takes two generic arrays, you need 81 copies! Even if they are the same type you still need 36 (!) copies.)

Not a good solution.

Re: GC Tuning Confessions of a Performance Engineer

#55

Post by TheLoneWolfling »

vitalyd wrote: post #51
evanpw wrote: post #33

Earlier quoted context omitted.

True, but you could do the same thing in any region of memory. The only thing that using "the stack" buys you is that there are a few special instructions to allocate / deallocate one machine word at a time (if you store the "top-of-heap" pointer in a specific register), and you get a bit more locality by virtue of return addresses being stored next to your locals and temporaries. (For x86 processors at least; maybe…

> True, but you could do the same thing in any region of memory. The only thing that using "the stack" buys you is that there are a few special instructions to allocate / deallocate one machine word at a time (if you store the "top-of-heap" pointer in a specific register) This is pretty much what a TLAB is in Hotspot JVM, except of course the pointer never moves backwards (once the TLAB is filled up, it retires, and…

Except that a TLAB is worse for cache than a stack.

Look at what happens when you call a bunch of small methods that each allocate a couple things. With a stack, everything stays in nice proximity automaticially. With a TLAB, you quickly fill the TLAB, get another one, get another one, and so on.

Effectively: the stack does any garbage collection that can be statically determined for free, keeping data in cache that actually matters, whereas with TLABs you need to wait for it to pass through GC before any of it is reused.

Re: GC Tuning Confessions of a Performance Engineer

#56

Post by pron »

vitalyd wrote: post #20
pron wrote: post #14

Earlier quoted context omitted.

> traversal is proportional to the number of pointers (rooted or dangling) in the heap and stack at the time of running. That's not quite how it works. Traversal is proportional to the number of pointers changed since the last collection (HotSpot's GCs do card marking). I do agree there are tradeoffs, but they're much more nuanced than that. There is certainly a footprint tradeoff, and there is a latency tradeoff (th…

A card in the card table covers more than one object, so the entire dirty card needs to be scanned. Card marks only help with old->young references, and stack based roots still need to be visited each time, irrespective of what happened in the last collection.

That's why I said proportional.

Re: GC Tuning Confessions of a Performance Engineer

#57

Post by vitalyd »

TheLoneWolfling wrote: post #55
vitalyd wrote: post #51

Earlier quoted context omitted.

> True, but you could do the same thing in any region of memory. The only thing that using "the stack" buys you is that there are a few special instructions to allocate / deallocate one machine word at a time (if you store the "top-of-heap" pointer in a specific register) This is pretty much what a TLAB is in Hotspot JVM, except of course the pointer never moves backwards (once the TLAB is filled up, it retires, and…

Except that a TLAB is worse for cache than a stack. Look at what happens when you call a bunch of small methods that each allocate a couple things. With a stack, everything stays in nice proximity automaticially. With a TLAB, you quickly fill the TLAB, get another one, get another one, and so on. Effectively: the stack does any garbage collection that can be statically determined for free, keeping data in cache that…

Right, that's why it bothers me when TLAB and stack are compared.

Re: GC Tuning Confessions of a Performance Engineer

#58

Post by vitalyd »

pron wrote: post #42
vitalyd wrote: post #19

Earlier quoted context omitted.

I really dislike the comparisons of TLAB to stack allocation. Stack, by its nature, is going to be hot in cache. TLAB, once filled up, will be retired and possibly assigned to a different thread. But even if it's not assigned, it's constantly moving forwards, and not revisiting the same space. You'd need prefetch to be perfect, and then on top of that, you'd need to make sure that by the time you go to allocate again…

While everything you say is 100% true, it is also a second-order effect, with a much lower magnitude than the primary GC performance behavior for short-lived objects. Obviously stack allocation is preferable to TLAB allocation (and there's no reason to allocate objects with stack scope on the heap), if only for the fact that it never triggers a collection. Nevertheless, Java allocation/collection of short-lived objec…

I'm not sure memory locality effects can be considered secondary, unless their effects are completely dwarfed by something else the app is doing (e.g. there's no point in discussing this topic for i/o bound workloads).

I don't think malloc/free should enter this conversation because languages that use malloc/free do so very infrequently, and for the cases where dynamic memory needs to be allocated frequently, they use specialized memory managers within the application. This is also subject to which allocator is used and what the application's allocation pattern is. There're suboptimal GC mechanics as well in some cases, such as CMS tenured space using free chunk lists with no compaction, so any young GC that requires promotion can possibly increase the young GC time because the GC needs to find appropriate free block size, and if there's fragmentation, this may take a while. Point being is that malloc/free vs GC isn't quite as clear cut on its own, nevermind that malloc/free aren't called that often. Generally speaking, though, if you can give GC ample headroom in terms of RAM, it'll have better throughput than incessant malloc/free use (which, I argue, is rare in properly written applications).

Re: GC Tuning Confessions of a Performance Engineer

#59

Post by vitalyd »

pjmlp wrote: post #32
bd_at_rivenhill wrote: post #30

Earlier quoted context omitted.

I think we will start to see less use of GCs in mobile going forward, especially now that Apple has deprecated GC in favor of automatic reference counting. The issues caused by the interaction between the memory hierarchy and the fragmentation that is naturally introduced by GC are difficult to measure, but nevertheless an important component of performance.

Lets not forget they did that, because they failed to produce a stable working GC for Objective-C.

A performant GC for mobile is inherently a much harder problem than for servers.

Re: GC Tuning Confessions of a Performance Engineer

#60

Post by kibwen »

jimrandomh wrote: post #34

Performance is nice, but correctness is more important. Complex programs written without any GC tend to have use-after-free and double-free vulnerabilities. Programs written using automatic reference counting only (without a way to catch cycles) tend to have memory leaks. If GC is killing performance, then that is a failure of the specific GC or programming language design. Don't blame the applications programmers fo…

  > that's a completely reasonable thing to want and 
  > something that language designers are capable of 
  > providing.
This is exactly what the designers of Rust have worked to provide: freedom from GC while maintaining guarantees against user-after-free and double-free, while allowing you to reference-count bits of data when you ask for it (RC leaks are rare in Rust--it's not trivial to create cycles--though still possible).
Post reply on HN