Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GC Tuning Confessions of a Performance Engineer

slideshare.net

Post reply on HN
Print view View original

81–90 of 98 posts

Re: GC Tuning Confessions of a Performance Engineer

#81

Post by pron »

banachtarski wrote: post #17
pron wrote: post #13

Earlier quoted context omitted.

> You will never beat a tuned system without GC with a GC system because of all the instructions necessary to traverse references and such. That's not at all how it works. The generational hypothesis means that most objects die young. Allocating them is a simple, uncontended pointer bump in the thread-local allocation buffer (as fast as stack allocation), and freeing them is free, as they are never traversed. They ar…

Comparing to malloc and free directly isn't very useful because that's pretty much the worst way to manage memory. Aside from memory arenas, you can just pool memory or preallocate too. I think concurrent data structures have very niche usage in practice (compared to say, a thread safe system). >> cache line misses hurt. > What does that have to do with GCs? Everything. Sure you can use a copying GC and hope related…

> I tell the compiler to lump them together and exploit the prefetcher wherever possible.

That's a memory layout issue. What does a GC have to do with that? .NET and Go already allow good memory layout control, and Java will, too, once it gets value types. Conversely, one could create a manually-managed language that doesn't support object/array embedding, either. Memory layout control and GCs are completely orthogonal issues.

Re: GC Tuning Confessions of a Performance Engineer

#82

Post by pron »

vitalyd wrote: post #70
pron wrote: post #65

Earlier quoted context omitted.

> I'm not sure memory locality effects can be considered secondary What is secondary isn't the TLAB/stack performance ratio, but that ratio vs malloc/TLAB. Also, I'm not sure why you think locality matters much here in the case of stack reuse. Within each frame, the stores always come first, and those go in the store buffer (and the reads are from the store buffer, too), so those are pretty benign cache misses. > whi…

Ok, we keep talking about malloc -- which malloc impl are you specifically referring to? There are many allocators out there these days, so let's be a bit more concrete. If not specific name, at least the class of allocator. Most of the common ones you'll find support thread-local allocation buffers, for starters. >Sure, it is rare in "well written applications", but how costly is it to write a well-written applicati…

> Most of the common ones you'll find support thread-local allocation buffers, for starters.

And what about concurrent deallocation?

> Have you, for example, looked at how postgresql manages memory? sqlite? redis? memcached?

Not too well (basically lots and lots of locking, much of it is very coarse-grained). Our spatial in-memory Java database (SpaceBase) offers an order-of-magnitude better performance in concurrent usage (and much better scalability with core number). We do over 200K transactions (with lots of contention) per second on a single 4-core laptop without breaking a sweat (concurrently with the application itself), and over a million on a large server (with some careful tuning).

But even for less super-concurrent databases, C++ databases don't outperform Java ones. In this benchmark, the Java databases (H2 and HSQLDB) almost always outperform MySQL and Postgres: http://www.h2database.com/html/performance.html (and I don't even know how the Java solutions handle concurrency, whether they do locking, optimistic locking or a clever combination, like SpaceBase).

In both cases, the amount of effort put into the Java solutions is orders of magnitude less than the C/C++ solutions.

> nginx? varnish?

Those are (virtually) read-only use cases. That is very easy to do concurrently no matter how you manage concurrency. The trick is concurrent writes, not reads.

Re: GC Tuning Confessions of a Performance Engineer

#83

Post by banachtarski »

pjmlp wrote: post #80
banachtarski wrote: post #69

Earlier quoted context omitted.

How many engineers work on, say, the Unreal Engine. The linux kernel? Windows? Do you think any of that stuff would work even marginally well running on the JVM (assuming the JVM had native driver support)? I don't quite follow your claims, nor do I put any stock in the tenure of your programming career. As for the off-shoring point, off-shoring is going to create questionable quality code regardless of the programmi…

> How many engineers work on, say, the Unreal Engine. The linux kernel? Windows? So you just provided me examples where developers tend to be highly skilled just to get a foot in the door. Yet, the CVE list gets updated regularly with memory corruption exploits for them. https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=double+free Or if you prefer, just for Linux https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=linux…

> So you just provided me examples where developers tend to be highly skilled just to get a foot in the door.

Yep these are people I want to work with :)

> Yet, the CVE list gets updated regularly with memory corruption exploits for them

Security is not the concern of every piece of software. You can harden the pieces you care about. Honestly, not all of us are doing crypto, and for that, the recommendation is to leverage an existing library and sandbox it anyways (regardless of memory management strategy).

> Actually, are you aware that some military weapon systems are being driven with JVMs?

God help them

> Another fun fact, Unreal Engine and Windows are written mostly in C++. A language considered too bloated and slow to be usable for anything serious by mainstream developers in the early 90's.

I was developing in C and C++ in the 90s. C++ is a strict subset of C. The criticisms made no sense then and they make no sense now. C++03 was arguable an small incremental improvement over C++98 but C++11 is a huge step function in usability without sacrificing performance (which Rust has taken many good cues from). The point is, if something bad is happening, I can look at the dissassembly and see exactly what's going on. I can't do this with Java, Python, Ruby, etc.

> Also both use automatic memory management on their systems. Unreal has a kind of GC library. Windows nowadays has COM almost everywhere with reference counting.

The scripting engine in Unreal supports GC if you want it. It's trivial to embed Lua or something. That's part of the point. For the non-performance sensitive bits, sure do whatever. The COM interface is terrible, and a necessary evil for those in the industry. There's a reason DirectX12 is looked forward to. I don't want Windows reference counting my things any more unless I tell it to.

> Anyway automatic memory management doesn't mean automatically a JVM, there are quite a few other ways.

... yes, I honestly can't imagine anyone in this thread would think otherwise. Heck, I've written a GC for a homebrew scripting language.

> Apparently some well known Fortune 100 companies and research institutes had another opinion.

My point is that how long you worked is sort of irrelevant in an actual academic discussion. Where you worked is just as irrelevant.

Re: GC Tuning Confessions of a Performance Engineer

#84

Post by pjmlp »

banachtarski wrote: post #83
pjmlp wrote: post #80

Earlier quoted context omitted.

> How many engineers work on, say, the Unreal Engine. The linux kernel? Windows? So you just provided me examples where developers tend to be highly skilled just to get a foot in the door. Yet, the CVE list gets updated regularly with memory corruption exploits for them. https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=double+free Or if you prefer, just for Linux https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=linux…

> So you just provided me examples where developers tend to be highly skilled just to get a foot in the door. Yep these are people I want to work with :) > Yet, the CVE list gets updated regularly with memory corruption exploits for them Security is not the concern of every piece of software. You can harden the pieces you care about. Honestly, not all of us are doing crypto, and for that, the recommendation is to lev…

> Yep these are people I want to work with :)

Which comes back to my point that only developers able to get hired at that level are able to do manual memory management in large scale.

And even then, they resort to automatic memory management at higher levels in their stacks.

> I can look at the dissassembly and see exactly what's going on. I can't do this with Java, Python, Ruby, etc.

Just get a commercial JVM compiler, JIT Watch, Intel Amplifier. All of them provide the option to show the generated machine code.

> There's a reason DirectX12 is looked forward to. I don't want Windows reference counting my things any more unless I tell it to.

Better stop developing for Windows then.

The Universal Windows Application model is the continuation of WinRT, an evolution of COM.

In case you missed the note, DirectX 12 is still based on COM.

Also the new User Driver Model is based on COM.

Re: GC Tuning Confessions of a Performance Engineer

#85

Post by banachtarski »

pjmlp wrote: post #84
banachtarski wrote: post #83

Earlier quoted context omitted.

> So you just provided me examples where developers tend to be highly skilled just to get a foot in the door. Yep these are people I want to work with :) > Yet, the CVE list gets updated regularly with memory corruption exploits for them Security is not the concern of every piece of software. You can harden the pieces you care about. Honestly, not all of us are doing crypto, and for that, the recommendation is to lev…

> Yep these are people I want to work with :) Which comes back to my point that only developers able to get hired at that level are able to do manual memory management in large scale. And even then, they resort to automatic memory management at higher levels in their stacks. > I can look at the dissassembly and see exactly what's going on. I can't do this with Java, Python, Ruby, etc. Just get a commercial JVM compil…

Yea still COM based but part of what is being removed is a huge swath of underlying automatic reference counting for a huge number of things. Constant buffers resources etc.

Sure you can look at machine code generated from a virtualized language but I'd have a tough time understanding it in the context of the whole runtime, let alone being able to practically do anything about it.

Re: GC Tuning Confessions of a Performance Engineer

#86

Post by pron »

pcwalton wrote: post #62
pron wrote: post #61

Earlier quoted context omitted.

I think we should look at the problem the other way around: it is manual memory management that is overkill because there are exactly four kinds of object memory scope: * stack scope * transaction scope (for some definition of transaction -- it can be, say, a frame in a game, or a request in a web server) * arbitrary (database or any shared data structure) * permanent For the stack scope, we have the stack. For the a…

> yes, vitalyd is going to mention traversals, but if the traversals are interesting, they point to objects in the arbitrary cost anyway I'm going to mention this too. I don't understand what you mean by "they point to objects in the arbitrary cost anyway". The fact of the matter is that you have to trace all objects at some point. > Now, I think it is far easier to manage a stack scope in a GCed environment than an…

> I'm going to mention this too.

What I mean is that either those refs are permanent -- in which case they won't be traced -- or arbitrary -- in which case it's really more of an "arbitrary" problem than a "permanent" one.

> Object pooling is just a (limited, error-prone, poorly-performing) form of manual memory management.

... And reference counting is just a (limited, error-prone, poorly-performing) form of garbage collection.

> Contention isn't a problem if you don't touch the reference counts much. Cycles aren't a problem if you don't have cycles.

But now you've added restrictions that are much more onerous than those of object-pools.

In any case, my main question is this: if absolute top performance is what you need, why not have a GC + arenas (like RTSJ) without ref counting/manual management? In my opinion, there's only one downside to that approach is RAM footprint. If that's your answer, then I agree (I think that that's pretty much the main reson not to use a GC).

Re: GC Tuning Confessions of a Performance Engineer

#87

Post by pjmlp »

banachtarski wrote: post #85
pjmlp wrote: post #84

Earlier quoted context omitted.

> Yep these are people I want to work with :) Which comes back to my point that only developers able to get hired at that level are able to do manual memory management in large scale. And even then, they resort to automatic memory management at higher levels in their stacks. > I can look at the dissassembly and see exactly what's going on. I can't do this with Java, Python, Ruby, etc. Just get a commercial JVM compil…

Yea still COM based but part of what is being removed is a huge swath of underlying automatic reference counting for a huge number of things. Constant buffers resources etc. Sure you can look at machine code generated from a virtualized language but I'd have a tough time understanding it in the context of the whole runtime, let alone being able to practically do anything about it.

> Sure you can look at machine code generated from a virtualized language but I'd have a tough time understanding it in the context of the whole runtime, let alone being able to practically do anything about it.

It is no different than using -S and check what happens for several code patterns.

Re: GC Tuning Confessions of a Performance Engineer

#88

Post by banachtarski »

pjmlp wrote: post #87
banachtarski wrote: post #85

Earlier quoted context omitted.

Yea still COM based but part of what is being removed is a huge swath of underlying automatic reference counting for a huge number of things. Constant buffers resources etc. Sure you can look at machine code generated from a virtualized language but I'd have a tough time understanding it in the context of the whole runtime, let alone being able to practically do anything about it.

> Sure you can look at machine code generated from a virtualized language but I'd have a tough time understanding it in the context of the whole runtime, let alone being able to practically do anything about it. It is no different than using -S and check what happens for several code patterns.

Look man, I don't know what to tell you except that you view code fundamentally differently. My point is that yes, obviously everything gets turned into x64/x86 assembly at some point, and that yes, I'm sure seeing it is very easy. But if I need to prevent a GC pause from happening at some point in the frame my hands are tied. Debating this is like pulling teeth, and I'm inclined to say that if you can't follow at this point, just use your GC everywhere.

Re: GC Tuning Confessions of a Performance Engineer

#89

Post by pjmlp »

banachtarski wrote: post #88
pjmlp wrote: post #87

Earlier quoted context omitted.

> Sure you can look at machine code generated from a virtualized language but I'd have a tough time understanding it in the context of the whole runtime, let alone being able to practically do anything about it. It is no different than using -S and check what happens for several code patterns.

Look man, I don't know what to tell you except that you view code fundamentally differently. My point is that yes, obviously everything gets turned into x64/x86 assembly at some point, and that yes, I'm sure seeing it is very easy. But if I need to prevent a GC pause from happening at some point in the frame my hands are tied. Debating this is like pulling teeth, and I'm inclined to say that if you can't follow at th…

This whole thread started by my statement that I haven't seen large projects without errors doing manual memory management.

I doubt very seriously that anyone can write code in large teams where Valgrind will state there aren't double frees, bad frees, or dangling pointers happening.

Or a run with Coverity will state everything is nice and shinning.

Once upon a time I had to write a tool in a well known particle accelerator research institute to track down C++ memory errors in a multi-thread environment for cluster algorithms used in data analysis. As one common problem on that specific team was plugins bringing the cluster down due to memory corruption.

So I also do know one or two things about manual memory management.

And with what I know, I rather use automatic memory management, be it in the form of GC, RC, affine types or dependent types.

Re: GC Tuning Confessions of a Performance Engineer

#90

Post by PopeOfNope »

banachtarski wrote: post #43
duaneb wrote: post #37

Earlier quoted context omitted.

Manual memory management (i.e. not GC) is a little disingenuous because it implies that it's entirely manual. In fact, there are many patterns (e.g. RAII) which make "brain dead" manual memory management possible. I'm not necessarily arguing this is a great way to go for a new team, but there are manual memory management strategies that can definitely scale. Among other things, it's not the only type of resource that…

Agreed. Sometimes, you don't even need RAII. You just let the stack unwind :) But yes, RAII can make it pretty braindead (which is good)

Do you have any book suggestions for learning modern C++ memory management techniques? I'm beginning to think maybe it's time to fill that gap in my programming knowledge.
Post reply on HN