Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Performance Impact of C++'s `final` Keyword

16bpp.net

Post reply on HN
Print view View original

251–260 of 385 posts

Re: The Performance Impact of C++'s `final` Keyword

#251

Post by neonsunset »

throwaway2037 wrote: post #249
neonsunset wrote: post #100

Earlier quoted context omitted.

.NET is a particularly bad case for this because it was a decade of few performance improvements, which caused a certain intuition to develop within the industry, then 6-8 years of significant changes each year (with most wins compressed to the last 4 years or so). Companies moving from .NET Framework 4.6/7/8 to .NET 8 experience a 10x average performance improvement, which naturally comes with rendering a lot of per…

.NET 4.6 to .NET 8 is a 10x "average" performance improvement. I find this hard to believe. In what scenarios? I tried to Google for it and found very little hard evidence.

In general purpose scenarios, particularly in codebases which have high amount of abstractions, use ASP.NET Core and EF Core, parse and de/serialize text with the use of JSON, Regex and other options, have network and file IO, and are deployed on many-core hosts/container images.

There are a few articles on msft devblogs that cover from-netframework migration to older versions (Core 3.1, 5/6/7):

- https://devblogs.microsoft.com/dotnet/bing-ads-campaign-plat...

- https://devblogs.microsoft.com/dotnet/microsoft-graph-dotnet...

- https://devblogs.microsoft.com/dotnet/the-azure-cosmos-db-jo...

- https://devblogs.microsoft.com/dotnet/one-service-journey-to...

- https://devblogs.microsoft.com/dotnet/microsoft-commerce-dot...

The tl;dr is depending on codebase the latency reduction was anywhere from 2x to 6x, varying per percentile, or the RPS was maintained with CPU usage dropping by ~2-6x.

Now, these are codebases of likely above average quality.

If you consider that moving 6 -> 8 yields another up to 15-30% on average through improved and enabled by default DynamicPGO, and if you also consider that the average codebase is of worse quality than whatever msft has, meaning that DPGO-reliant optimizations scale way better, it is not difficult to see the 10x number.

Keep in mind that while particular regular piece of enterprise code could have improved within bounds of "poor netfx codegen" -> "not far from LLVM with FLTO and PGO", the bottlenecks have changed significantly where previously they could have been in lock contention (within GC or user code), object allocation, object memory copying, e.g. for financial domains - anything including possibly complex Regex queries on imported payment reports (these alone have now difference anywhere between 2 and >1000[0]), and for pretty much every code base also in interface/virtual dispatch for layers upon layers of "clean architecture" solutions.

The vast majority of performance improvements (both compiler+gc and CoreLib+frameworks), which is difficult to think about, given it was 8 years, address the above first and foremost. At my previous employer the migration from NETFX 4.6 to .NET Core 3.1, while also deploying to much more constrained container images compared to beefy Windows Server hosts, reduced latency of most requests by the same factor of >5x (certain request type went from 2s to 350ms). It was my first wow moment when I decided to stay with .NET rather than move over to Go back then (was never a fan of syntax though, and other issues, which subsequently got fixed in .NET, that Go still has, are not tolerable for me).

[0] Cumulative of

https://devblogs.microsoft.com/dotnet/regex-performance-impr...

https://devblogs.microsoft.com/dotnet/regular-expression-imp...

https://devblogs.microsoft.com/dotnet/performance-improvemen...

Re: The Performance Impact of C++'s `final` Keyword

#252

Post by kccqzy »

jeffbee wrote: post #99
gpderetta wrote: post #72

Earlier quoted context omitted.

I haven't looked at the code, but if you have multiple leaves, even marking all of them as final won't help if the call is through a base class.

Yeah the practical cases for devirtualization are when you have a base class, a derived class that you actually use, and another derived class that you use in tests. For your release binary the tests aren't visible so that can all be devirtualized. In cases where you have Dog and Goose that both derive from Animal and then you have std::vector , what is the compiler supposed to do?

The compiler simply knows that the actual dynamic type is Animal because it is not a pointer. You need Animal* to trigger all the fun virtual dispatch stuff.

Re: The Performance Impact of C++'s `final` Keyword

#253

Post by mynameisnoone »

tombert wrote: post #26

I don't do much C++, but I have definitely found that engineers will just assert that something is "faster" without any evidence to back that up. Quick example, I got in an argument with someone a few years ago that claimed in C# that a `switch` was better than an `if(x==1) elseif(x==2)...` because switch was "faster" and rejected my PR. I mentioned that that doesn't appear to be true, we went back and forth until I…

Yep. "Profiling or it didn't happen." The issue is that it's essentially impossible for even the most neckbeard of us to predict with a high degree of accuracy and precision the performance on modern systems impact of change A vs. change B due to the unpredictable nature of the many variables that are difficult to control including compiler optimization passes, architecture gotchas (caches, branch misses), and interplay of quirks on various platforms. Therefore, irreducible and necessary work to profile the differences become the primary viable path to resolving engineering decision points. Hopefully, LLMs now and in the future will be able to help build out boilerplate roughly in the direct of creating such profiling benchmarks and fixtures.

PS: I'm presently revisiting C++14 because it's the most universal statically-compiled language to quickly answer interview problems. It would be unfair to impose Rust, Go, Elixir, or Haskell on an interviewer software engineer.

Re: The Performance Impact of C++'s `final` Keyword

#254

Post by drivebycomment »

sfink wrote: post #18

tldr: sprinkled a keyword around in the hopes that it "does something" to speed things up, tested it, got noisy results but no miraculous speedup. I started skimming this article after a while, because it seemed to be going into the weeds of performance comparison without ever backing up to look at what the change might be doing. Which meant that I couldn't tell if I was going to be looking at the usual random noise…

+1. On modern hardware and software systems, performance is effectively stochastic to some degree, as small random perturbations to the input (code, data, environments, etc) can have arbitrary effects for the performance. This means you can't draw a direct causal chain / mechanism from what you changed to the performance change - when it matters, you do need to do a deeper analysis and investigation to find the actual and full causal chain. I.e. a correlation is not a causation, and especially more so on modern hardware and software systems.

Re: The Performance Impact of C++'s `final` Keyword

#256

Post by EasyMark »

qsdf38100 wrote: post #137
phire wrote: post #117

Earlier quoted context omitted.

Jumps/calls are actually be pretty cheap with modern branch predictors. Even indirect calls through vtables, which is the opposite of most programmers intuition. And if the devirtualisation leads to inlining, that results in code bloat which can lower performance though more instruction cache misses, which are not cheap. Inlining is actually pretty evil. It almost always speeds things up for microbenchmarks, as such…

"Inlining is actually pretty evil". No it's not. Except if you __force_inline__ everything, of course. Inlining reduces the number of instructions in a lot of cases. Especially when things are abstracted and factored with lot of indirections into small functions that calls other small functions and so on. Consider a 'isEmpty' function, which dissolves to 1 cpu instruction once inlined, compared with a call/save reg/c…

doesn't the compiler usually do well enough that you really only need to worry about time critical sections of code? Even then you could go in and look at the assembler and see if it's being inlined, no?

Re: The Performance Impact of C++'s `final` Keyword

#258

Post by rerdavies »

neonsunset wrote: post #251
throwaway2037 wrote: post #249

Earlier quoted context omitted.

.NET 4.6 to .NET 8 is a 10x "average" performance improvement. I find this hard to believe. In what scenarios? I tried to Google for it and found very little hard evidence.

In general purpose scenarios, particularly in codebases which have high amount of abstractions, use ASP.NET Core and EF Core, parse and de/serialize text with the use of JSON, Regex and other options, have network and file IO, and are deployed on many-core hosts/container images. There are a few articles on msft devblogs that cover from-netframework migration to older versions (Core 3.1, 5/6/7): - https://devblogs.mi…

Cheating.

All of the 6x performance improvement cases seem to be related to using the .net based Kestrel web server instead of IIS web server, which requires marshalling and interprocess communication. Several of the 2x gains appear to be related to using a different database backend. Claims that regex performance has improved a thousand-fold.... seem more troubling than cause for celebration. Were you not precompiling your regex's in the older code? That would be a bug.

Somewhere in there, there might be 30% improvements in .net codegen (it's hard to tell). Profile Guided Optimization (PGO) seems to provide a 35% performance improvement over older versions of .net with PGO disabled. But that's dishonest. PGO was around long before .net Core. And claiming that PGO will provide 10x performance because our code is worse than Microsoft's code insults both our code and our intelligence.

Re: The Performance Impact of C++'s `final` Keyword

#259

Post by adrianN »

doctor_phil wrote: post #44
wvenable wrote: post #41

Earlier quoted context omitted.

In my opinion, the only things that really matter are algorithmic complexity and readability. And even algorithmic complexity is usually only an issue a certain scales. Whether or not an 'if' is faster than a 'switch' is the micro of micro optimizations -- you better have a good reason to care. The question I would have for you is was your bunch of ifs more readable than a switch would be.

But a switch and an if-else *is* a matter of algorithmic complexity. (Well, at least could be for a naive compiler). A switch could be converted to a constant time jump, but the if-else would be trying each case linearly.

Both the switch and the if have O(1) instructions, so both are the same from an algorithmic complexity perspective.

Re: The Performance Impact of C++'s `final` Keyword

#260

Post by usefulcat »

EasyMark wrote: post #256
qsdf38100 wrote: post #137

Earlier quoted context omitted.

"Inlining is actually pretty evil". No it's not. Except if you __force_inline__ everything, of course. Inlining reduces the number of instructions in a lot of cases. Especially when things are abstracted and factored with lot of indirections into small functions that calls other small functions and so on. Consider a 'isEmpty' function, which dissolves to 1 cpu instruction once inlined, compared with a call/save reg/c…

doesn't the compiler usually do well enough that you really only need to worry about time critical sections of code? Even then you could go in and look at the assembler and see if it's being inlined, no?

I find that gcc and clang are so aggressive about inlining that it's usually more effective to tell them what not to inline.

In a moderately-sized codebase I regularly work on, I use __attribute__((noinline)) nearly ten times as often as __attribute__((always_inline)). And I use __attribute__((cold)) even more than noinline.

So yeah, I can kind of see why someone would say inlining is 'evil', though I think it's more accurate to say that it's just not possible for compilers to figure out these kinds of details without copious hints (like PGO).

Post reply on HN