Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Performance Impact of C++'s `final` Keyword

16bpp.net

Post reply on HN
Print view View original

371–380 of 385 posts

Re: The Performance Impact of C++'s `final` Keyword

#371

Post by pcvarmint »

Each of the test cases measured needs to be run at least 3 times in a row, to warm caches (not just CPU but OS too) and to detect and remove noise.

In fact, I would run the same test repeatedly, keeping track of the k fastest times (k being ~3-7), and only stopping when the first and the kth fastest times are within a certain tolerance (as low as 1%). This ensures repeatability.

One sample of performance data for each test is not enough. This study provides no new insights.

Performance analyst

Re: The Performance Impact of C++'s `final` Keyword

#372

Post by simonask »

akoboldfrying wrote: post #306
simonask wrote: post #280

Earlier quoted context omitted.

Actually, the compiler can only implicitly devirtualize under very specific circumstances. For example, it cannot devirtualize if there was previously a non-inlined call through the same pointer. The reason is placement new. It is legal (given that certain invariants are upheld) in C++ to say `new(this) DerivedClass`, and compilers must assume that each method could potentially have done this, changing the vtable poi…

Fascinating, though a little sad. Are there any important kinds of behaviour that can only be implemented via this `new(this) DerivedClass` chicanery? Because if not, it seems a shame to make the optimiser pay such a heavy price just to support it.

Presumably there is some arcane trick that somebody will argue is only implementable in this way, but I would personally never let such code through review.

Re: The Performance Impact of C++'s `final` Keyword

#373

Post by simonask »

rurban wrote: post #367
simonask wrote: post #277

Earlier quoted context omitted.

An extra indirection (indirect call versus direct call) is practically nothing on modern hardware. Branch predictors are insanely good, and this isn't something you generally have to worry about. Inlining is by far the most impactful optimization here, because it can eliminate the call altogether, and thus specialize the called function to the callsite, lifting constants, hoisting loop variables, etc.

C++ vtables need 2 levels of indirection. See the asm or decompile it with ghidra. First the vtable field, and then the method field. Of course you have to worry about pointer chasing, when you can easily avoid it. Either via a switch to a single indirection (by passing method pointers around) or inlining with final. Or other compile-time specialization.

Show me the receipts. :-)

In general it takes a significant amount of nondeterministic pointer chasing to fool modern branch predictors. Decades of research have been put into optimizing the hardware for languages like C++ and Java, both of which exhibit a lot of pointer chasing.

Re: The Performance Impact of C++'s `final` Keyword

#374

Post by simonask »

silvestrov wrote: post #303
simonask wrote: post #277

Earlier quoted context omitted.

An extra indirection (indirect call versus direct call) is practically nothing on modern hardware. Branch predictors are insanely good, and this isn't something you generally have to worry about. Inlining is by far the most impactful optimization here, because it can eliminate the call altogether, and thus specialize the called function to the callsite, lifting constants, hoisting loop variables, etc.

"is practically nothing on modern hardware" if the data is already present in the L2 cache. Random RAM access that stalls execution is expensive. My guess is this is why he didn't see any speedup: all the code could fit inside the L2 cache, so he did not have to pay for RAM access for the deference. The number of different classes is important, not the number of objects as they have the same small number of vtable po…

Both the number of objects (dcache) and the number of classes (icache) are significant, as well as the size of both, but yeah. It's pretty rare to have extremely wide class hierarchies, though. You really have to go out of your way to run into significant icache misses.

Re: The Performance Impact of C++'s `final` Keyword

#375

Post by Bene592 »

magnat wrote: post #124

> I created a "large test suite" to be more intensive. On my dev machine it needed to run for 8 hours. During such long and compute-intensive tests, how are thermal considerations mitigated? Not saying that this was case here, but I can see how after saturating all cores for 8 hours, the whole PC might get hot to the point CPU starts throttling, so when you reboot to next OS or start another batch, overall performanc…

8 hours should be enough to let the temperatures settle

Re: The Performance Impact of C++'s `final` Keyword

#376

Post by moring »

somenameforme wrote: post #327
EasyMark wrote: post #256

Earlier quoted context omitted.

doesn't the compiler usually do well enough that you really only need to worry about time critical sections of code? Even then you could go in and look at the assembler and see if it's being inlined, no?

I find the Unreal Engine source to be a reasonable reference for C++ discussions, because it runs just unbelievably well for what it does, and on a huge array of hardware (and software). And it's explicit with inlining, other hints, and even a million things that could be easily called micro-optimizations, to a somewhat absurd degree. So I'd take away two conclusions from this. The first is that when building a code…

> The second is that I think the saying 'premature optimization is the root of all evil' is the root of all evil.

The greater evil is putting a one-sentence quote out of context:

""" There is no doubt that the grail of efficiency leads to abuse. Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil.

Yet we should not pass up our opportunities in that critical 3%. A good programmer will not be lulled into complacency by such reasoning, he will be wise to look carefully at the critical code; but only after that code has been identified. It is often a mistake to make a priori judgments about what parts of a program are really critical, since the universal experience of programmers who have been using measurement tools has been that their intuitive guesses fail. After working with such tools for seven years, I've become convinced that all compilers written from now on should be designed to provide all programmers with feedback indicating what parts of their programs are costing the most; indeed, this feedback should be supplied automatically unless it has been specifically turned off. """

Re: The Performance Impact of C++'s `final` Keyword

#377

Post by account42 »

adrianN wrote: post #368
account42 wrote: post #337

Earlier quoted context omitted.

Const affects code generation when used on variables . If you have a `const int i` then the compiler can assume that i never changes. But you're right that this does not hold true for const pointers or references. > What actually may help is __attribute__((pure)) and __attribute__((const)), but I don't see them often in real code (unfortunately). It's disppointing that these haven't been standardized. I'd prefer diff…

Do you have an example where a const on a variable changes codegen? I would be surprised if the compiler couldn't figure out variable constness itself.

Sure, in the following example the compiler is able to propagate the constant to the return statement with const in f1 but needs to load it back from the stack without const in f0:

https://godbolt.org/z/6ebrbaM7b

In general, whenever you call a function that the compiler cannot inspect (because it is in another TU) and the compiler cannot prove that that function doesn't have any reference to your variable it has to assume that the function might change your variable. Only passing a const reference won't help you here because it is legal to cast away constness and modify the variable unless the original variable was const.

I wish that const meant something on reference or pointers and you had to do something more explicit like a mutable member to allow modifying a variable. But even that would not help if the compiler can't prove that a non-const pointer hasn't escaped somehow. You could add __attribute__((pure)) to the function to help the compiler but that is a lot stricter so can't always be used.

Re: The Performance Impact of C++'s `final` Keyword

#378

Post by account42 »

bayindirh wrote: post #369
account42 wrote: post #337

Earlier quoted context omitted.

Const affects code generation when used on variables . If you have a `const int i` then the compiler can assume that i never changes. But you're right that this does not hold true for const pointers or references. > What actually may help is __attribute__((pure)) and __attribute__((const)), but I don't see them often in real code (unfortunately). It's disppointing that these haven't been standardized. I'd prefer diff…

> If you have a `const int i` then the compiler can assume that i never changes. Plus, you can’t even compile your code if you try to modify a const variable.

This isn't guaranteed: Modification after const cast on a const variable is ill formed but the compiler is not required to diagnose it - an generally it can't because it doesn't know what your reference/pointer points to.

Re: The Performance Impact of C++'s `final` Keyword

#379

Post by somenameforme »

moring wrote: post #376
somenameforme wrote: post #327

Earlier quoted context omitted.

I find the Unreal Engine source to be a reasonable reference for C++ discussions, because it runs just unbelievably well for what it does, and on a huge array of hardware (and software). And it's explicit with inlining, other hints, and even a million things that could be easily called micro-optimizations, to a somewhat absurd degree. So I'd take away two conclusions from this. The first is that when building a code…

> The second is that I think the saying 'premature optimization is the root of all evil' is the root of all evil. The greater evil is putting a one-sentence quote out of context: """ There is no doubt that the grail of efficiency leads to abuse. Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have…

Indeed, but I think even that advice, with context, is pretty debatable. Obviously one should prioritize critical sections, but completely ignoring those "small efficiencies" is certainly a big part of how we got to where we are today in software performance. A 10% jump in performance is huge; whether that comes from a single 10% jump, or a hundred 0.1% jumps - it's exactly the same!

So referencing something in particular from Unreal Engine, they actually created a caching system for converting between a quaternion and a rotator (euler rotation)! Obviously that sort of conversion isn't going to, in a million years, be even close to a bottleneck. That conversion is quite cheap on modern hardware, and so that caching system probably only gives the engine one of those 0.1% boosts in performance. But there are literally thousands of these "small efficiencies" spread all throughout the code. And it yields a final product that runs dramatically better than comparable engines.

Re: The Performance Impact of C++'s `final` Keyword

#380

Post by phire »

rurban wrote: post #367
simonask wrote: post #277

Earlier quoted context omitted.

An extra indirection (indirect call versus direct call) is practically nothing on modern hardware. Branch predictors are insanely good, and this isn't something you generally have to worry about. Inlining is by far the most impactful optimization here, because it can eliminate the call altogether, and thus specialize the called function to the callsite, lifting constants, hoisting loop variables, etc.

C++ vtables need 2 levels of indirection. See the asm or decompile it with ghidra. First the vtable field, and then the method field. Of course you have to worry about pointer chasing, when you can easily avoid it. Either via a switch to a single indirection (by passing method pointers around) or inlining with final. Or other compile-time specialization.

Though the branch predictor can chew though both layers of indirection. It can actually start fetching code from the function (and even executing it) before it even reads the function pointer from the vtable.

Though, that assumes a correct prediction. But modern branch predictors are really good, they can track and correctly predict hundreds (if not thousands) of indirect calls, taking into account the history of the last few branches (so it can even get an idea of what class is currently being executed, and make branch predictions based on that). Modern branch predictors do a really good job at chewing up indirect branches in hot sequences of code.

Virtual functions are probably the most harmful for warm code. We are talking about code that's executed too often to be considered cold code, but not often enough to stick around in the branch predictors' cache, executed only a few hundred times a second. It's a death by a thousand cuts type thing. And that's where devirtualisation will help the most...

As long as you don't go too far with the inlining and start causeing icache misses with code bloat. In an ideal would the compiler would inline enough to devirtualise the class, but not necessarily inline the actual function (unless they are small, or only called from one place)

Post reply on HN