Live data from Hacker News

The Garbage Collection Handbook, 2nd Edition

routledge.com

161–170 of 174 posts

Re: The Garbage Collection Handbook, 2nd Edition

#161

Earlier quoted context omitted.

I will say that you seem to be stating "smearing that over total execution time tends to give better results" without proof as well. I certainly don't think using atomic operations everywhere and converting every pointer read operation into a write operation is efficient. Yes RC has smaller minheap sizes than tracing GC, but garbage collection is fundamentally a time-space tradeoff and RC is highly optimized for memo…

As I’ve stated repeatedly, RC is super cheap. Seriously. You can do about several billion of them per second. Your malloc/free call is going to be more expensive. Sure. Atomic counters are relatively expensive. But I’m good designs there’s very few of them. And they easily show up in hotspots if they’re a problem and you fix your object model. The problem with tracing GC is that you have no way to fix it. Most langua…

> RC is super cheap. Seriously. You can do about several billion of them per second.

Right. You bump allocate faster, however. RC is an additional operation to the allocation request. Given naive RC can't move objects, it necessarily needs a free-list allocator. Free-list allocators can allocate pretty fast (for the most common object sizes), but can't reach the speeds of bump allocators. Furthermore, bump allocators have better locality of reference than free-list allocators.

Also I've never questioned you can't do non-atomic increments/decrements efficiently. They are exceptionally fast. However you are still converting every pointer read operation into a pointer write operation. The rule of thumb is that there are generally 10x more pointer read operations in a program than pointer write operations. This is why read barriers are also considered more costly than write barriers.

> I did actually provide proof by the way. Apple’s phones use half the RAM as Android and are at least as equally fast even if you discount better HW.

I don't really agree with this statement. There are way too many unknown/unaccounted variables. It could be better hardware, maybe Android has a terrible architecture, and could just be as you said that Swift RC is genuinely better than ART GC or it can be whatever. Point is that it's not a scientific comparison. We don't know if the benefits in iOS come from RC. And we wont be able to know unless we have two systems where all parameters are the exact same _except_ one is using RC and another is using tracing GC, with both systems ran on the same hardware and on the same set of benchmarks.

> That’s only kind of true. Good allocators seem to mitigate this problem quite effectively

Yes. Note that mimalloc literally has a concept of "deferred frees" effectively emulating GC as otherwise freeing an object can result in an unbounded recursive free call (for example, dropping a large linked list).

> I’m just saying that good memory management (made fairly easy in Rust) is always going to outperform tracing GC the same way optimized assembly will outperform the compiler.

Sure. The perfect memory manager is omniscient and knows exactly when an object is not required and will free it. But unfortunately we don't have perfect memory managers yet. So yes I agree in principle, but we have to be pragmatic.

> And I can’t belabor this point enough - languages with RC use it rarely as shared ownership is rarely needed and you can typically minimize where you use it.

I feel like that entirely depends on the problem domain? I don't know where you're getting the "shared ownership is rarely needed" from? Maybe this is true for your problem domain but may not be for others. And if it's true for yours, then great! You can use RC and optimize your programs that way.

> A tracing garbage collector still needs to do atomic reads of data structures which potentially requires cross cpu shoot downs.

Sure. GC metadata needs to be accessed and updated atomically. 100% agree with you. The order of magnitude of those operations is likely much less than what you would get with naive RC though.

> as Swift and ObjC demonstrate, it’s generally good enough without any serious performance implications (throughput or otherwise).

In one of my comments about Swift in this thread, the two papers I linked see up to 80% of the benchmark execution time being dominated by ARC operations! I will note that the papers are around 6-7 years old so the situation might have drastically changed since then, but I haven't personally found many new/contemporary evaluations of Swift RC.

Re: The Garbage Collection Handbook, 2nd Edition

#162
post #160

Earlier quoted context omitted.

> Your single threaded RC will still have to write back to memory I think you mean mem or cache, and there's a good chance it will remain in cache and not be flushed to ram for short lived objects. > no one thinks that incrementing an integer is the slow part — destroying cache is. agreed

If you write to cache, then depending on architecture that change has to be made visible to every other thread as well. Reading is not subject to such a constraint.

MESI cache coherency (and its derivatives) [1] means that you can have exclusive writes to cache if and only if no other core tries to access that data. I would think most if not all microarchitectures have moved to MESI (or equivalent) cache coherency protocols as they avoid unnecessary writes to memory.

[1]: https://en.wikipedia.org/wiki/MESI

Re: The Garbage Collection Handbook, 2nd Edition

#163
post #143

Earlier quoted context omitted.

So fast that Apple Silicon introduced specific memory instructions to handle ARC counters.

It did not do this, standard atomics are relatively fast because it's a unified memory system.

ARM also has an option for weaker consistency atomics which help with speed (and ARC does take advantage of this but not something Apple sped up specifically extra afaik in silicon)

Re: The Garbage Collection Handbook, 2nd Edition

#164
post #142

Earlier quoted context omitted.

Because tracing GCs can solve referential loops which RC can’t. So at the language level where you have to handle all sorts of programs written by programmers of varying quality (+ mistakes) a tracing GC gives better predictable memory usage performance across a broader range of programs. Seriously. A single threaded reference counter is super cheap. Cross thread reference counts shouldn’t be used and I think are an…

Obviously the 3 objects you allocate with shared_ptr in C++ won’t be a performance bottleneck, but then we are not comparing apples to oranges. Your single threaded RC will still have to write back to memory, no one thinks that incrementing an integer is the slow part — destroying cache is.

Even when I had this in the hot path 10 years ago and was owning hundreds of objects in a particle filter, handing out ownership copies and creating new ones ended up taking ~5% (ie making it a contiguous vector without any shared_ptr). It can be expensive but in those case you probably shouldn’t be using shared_ptr.

Oh, and the cost of incrementing an integer by itself (non atomically) is stupid fast. Like you can do a billion of them per second. The CPU doesn’t actually write that immediately to RAM and you’re not putting a huge amount of extra cache pressure vs all the other things your program is doing normally.

Re: The Garbage Collection Handbook, 2nd Edition

#165
post #160

Earlier quoted context omitted.

> Your single threaded RC will still have to write back to memory I think you mean mem or cache, and there's a good chance it will remain in cache and not be flushed to ram for short lived objects. > no one thinks that incrementing an integer is the slow part — destroying cache is. agreed

If you write to cache, then depending on architecture that change has to be made visible to every other thread as well. Reading is not subject to such a constraint.

Only for atomics. Single-threaded RC counts are fine and WebKit uses hybrid RC where you use an atomic shared ptr to share across threads and then you downgrade it to a non atomic version in-thread (and can hand out another atomic copy at any time).

Atomics are rarely needed as you should really try to avoid sharing ownership across threads and instead change your design to avoid that if you can.

Re: The Garbage Collection Handbook, 2nd Edition

#166
post #160

Earlier quoted context omitted.

If you write to cache, then depending on architecture that change has to be made visible to every other thread as well. Reading is not subject to such a constraint.

MESI cache coherency (and its derivatives) [1] means that you can have exclusive writes to cache if and only if no other core tries to access that data. I would think most if not all microarchitectures have moved to MESI (or equivalent) cache coherency protocols as they avoid unnecessary writes to memory. [1]: https://en.wikipedia.org/wiki/MESI

Sure, but for shared objects you can’t allow data races.

Re: The Garbage Collection Handbook, 2nd Edition

#167

Earlier quoted context omitted.

Correct. But it’ll do it within functions in the same module. > but it's deterministic after compilation time so I wouldn't call it a runtime optimization. What do you mean? My understanding is that autoreleasepool is 100% at runtime. The compiler is not involved afaik except to know to register autorelease with the currently installed pool.

> What do you mean? My understanding is that autoreleasepool is 100% at runtime. The compiler is not involved afaik except to know to register autorelease with the currently installed pool. The compiler emits calls that always put something in the autorelease pool or always don't; there's no smart decisions at runtime that skips it or make the ordering of releases nondeterministic. A garbage collector runs whenever i…

Oh sure. You have to explicitly indicate which resources should go to the autoreleasepool. It’s not as magic as ARC. It’s also generally fallen out of favor at Apple as far as I could tell.

Re: The Garbage Collection Handbook, 2nd Edition

#168
post #166

Earlier quoted context omitted.

MESI cache coherency (and its derivatives) [1] means that you can have exclusive writes to cache if and only if no other core tries to access that data. I would think most if not all microarchitectures have moved to MESI (or equivalent) cache coherency protocols as they avoid unnecessary writes to memory. [1]: https://en.wikipedia.org/wiki/MESI

Sure, but for shared objects you can’t allow data races.

Objects shared across threads. Most don’t need to be.

Re: The Garbage Collection Handbook, 2nd Edition

#169

On a related note, I found ART's implementation of Concurrent copying and compaction GC to be pretty novel. Here's a nice write up detailing how handling page faults in userspace come in handy to accomplish that: https://www.tdcommons.org/cgi/viewcontent.cgi?article=4751&c... (pdf) / https://web.archive.org/web/20230216233459/https://www.tdcom... For context, here's a brief overview of the evolution of the Android Ru…

Certainly interesting, but there are no performance numbers mentioned in the white paper comparing userfaultfd to read barriers. So the actual benefit to switching to userfaultfd is unknown (at least in publically accessible documents -- I'm sure Google has done internal performance evaluations). Using page faults (and/or page protection) to perform compaction instead of barriers is a pretty old technique (see the 19…

What is the actual perf benefit of userfaultd (e.g. for write protection)? It sounds interesting but its unclear to me why it would be any faster than a signal handler. Is it just a simpler code path within the kernel? Or is it that the hardware is configured to directly jump to a user space handler without any kernel intervention?

Re: The Garbage Collection Handbook, 2nd Edition

#170

Earlier quoted context omitted.

Certainly interesting, but there are no performance numbers mentioned in the white paper comparing userfaultfd to read barriers. So the actual benefit to switching to userfaultfd is unknown (at least in publically accessible documents -- I'm sure Google has done internal performance evaluations). Using page faults (and/or page protection) to perform compaction instead of barriers is a pretty old technique (see the 19…

What is the actual perf benefit of userfaultd (e.g. for write protection)? It sounds interesting but its unclear to me why it would be any faster than a signal handler. Is it just a simpler code path within the kernel? Or is it that the hardware is configured to directly jump to a user space handler without any kernel intervention?

Well page protection is expensive which is why the predominant way to implement concurrent copying/compaction until recently was using read barriers (and still is -- ART seems to be an exception). I will mention that concurrent compacting GCs wouldn't use userfaultfd for write protection, but for read protection (it effectively takes over the job of a read barrier).

I personally don't know too much about userfaultfd and how it works internally, but my best guess is that it bypasses heavyweight kernel data-structures and lets the user application handle the page fault. It is obviously better than simply using mprotect, but it is not immediately clear why it would be better than a read barrier (other than code size considerations, which honestly doesn't sound like much of a big deal as the code handling userfaultfd also needs to be brought into the instruction cache).

I did find this kernel doc about userfaultfd [1] which might be interesting to read if you're interested (it also does mention that userfaultfd doesn't lock some internal kernel data-structures which gives it a better performance than simply using mprotect, but the implementation details are a bit sparse).

[1]: https://docs.kernel.org/admin-guide/mm/userfaultfd.html

Post reply on HN