The cost of dynamic vs. static dispatch in C++ (2013) • Hacker News Board

Re: The cost of dynamic vs. static dispatch in C++ (2013)

#21

Post by adrusi » Sat, Feb 28, 2015, 11:31 PM UTC

MichaelGG wrote: post #19

twoodfin wrote: post #16

As the article demonstrates, in anything other than trivial microbenchmarks it's the inability to inline across a virtual call that will cost you. Inlining is the single most fundamental optimization for a modern C++ compiler. It enables the 0-cost abstractions for which C++ is rightly famous.

Why shouldn't it be possible to inline virtual calls? The CLR folks were complaining about this, too. But it seems rather straightforward to figure out the actual implementation that's commonly called and inline it, perhaps with a guard for uncommon cases. And I wouldn't be surprised if many apps end up with only a single actual implementation, so the entire virtual overhead can just be dropped.

Because c++'s type system doesn't make it possible to determine the actual implementation in most cases. Concepts would allow rust-style monomorphization, but they aren't part if the standard yet.

Re: The cost of dynamic vs. static dispatch in C++ (2013)

#22

Post by tezka » Sat, Feb 28, 2015, 11:34 PM UTC

jjaredsimpson wrote: post #4

kspiteri wrote: post #2

If you can use templates, the decision of which function to call can be done at compile time and you don't need virtual calls. Virtual calls are useful when the decision must be done at runtime.

Article just seems like "I misused some feature X and it was slower than something else."

no it's not. it's more like, the most common dispatch pattern used in C++ (and most other OO languages) as a factor of 7x performance penalty over the less widely embraced alternative. The popularity of the former methods is partly due to the ignorance of programmers to its performance implications. I have seen huge code bases that had to be retired because of their over reliance on OO, and dynamic dispatch.

Re: The cost of dynamic vs. static dispatch in C++ (2013)

#23

Post by Jimmy » Sat, Feb 28, 2015, 11:57 PM UTC

stinos wrote: post #13

zsombor wrote: post #3

In 2015 clang & gcc have a more or less working devirtualization feature that eliminates the virtual overhead in a simple benchmark such as this. The compiler sees all classes and knows that a particular interface is implemented by only one class, so it simply elides the virtual table lookup. Add that features such as speculative devirtualization that can remove a surprisingly large number of virtual calls in an esta…

If C++ will exist in 10 years from now Apart from it being quite an interesting do-it-all type of language, solely seeing the huge amounts of existing codebases in 'slow' fields like production/manufacturing industries I am pretty sure it will.

Even if new applications stopped being written in C++ today, the amount of legacy code written in C++ would ensure that there would be jobs for C++ programmers for many decades to come. But of course, people will almost certainly still be writing new C++ code 10 years from now.

Re: The cost of dynamic vs. static dispatch in C++ (2013)

#24

Post by kazinator » Sun, Mar 01, 2015, 1:27 AM UTC

twoodfin wrote: post #16

As the article demonstrates, in anything other than trivial microbenchmarks it's the inability to inline across a virtual call that will cost you. Inlining is the single most fundamental optimization for a modern C++ compiler. It enables the 0-cost abstractions for which C++ is rightly famous.

Some virtual calls cannot be converted to static calls and inlined, because dynamic dispatch solves a problem which static calls do not solve. In that case, it is a bicycle versus canoe comparison.

C++ does a good job of inlining those calls to virtual functions that can be inlined: like when a virtual is invoked on an object whose exact type is obvious from the scope.

Re: The cost of dynamic vs. static dispatch in C++ (2013)

#25

Post by kazinator » Sun, Mar 01, 2015, 1:30 AM UTC

adrusi wrote: post #21

MichaelGG wrote: post #19

Earlier quoted context omitted.

Why shouldn't it be possible to inline virtual calls? The CLR folks were complaining about this, too. But it seems rather straightforward to figure out the actual implementation that's commonly called and inline it, perhaps with a guard for uncommon cases. And I wouldn't be surprised if many apps end up with only a single actual implementation, so the entire virtual overhead can just be dropped.

Because c++'s type system doesn't make it possible to determine the actual implementation in most cases. Concepts would allow rust-style monomorphization, but they aren't part if the standard yet.

None of this matters if you create the "flexible joints" in the right places, where you get the benefit of dispatch, without making hundreds of millions of calls per second to it.

Re: The cost of dynamic vs. static dispatch in C++ (2013)

#26

Post by kazinator » Sun, Mar 01, 2015, 1:34 AM UTC

tezka wrote: post #22

jjaredsimpson wrote: post #4

Earlier quoted context omitted.

Article just seems like "I misused some feature X and it was slower than something else."

no it's not. it's more like, the most common dispatch pattern used in C++ (and most other OO languages) as a factor of 7x performance penalty over the less widely embraced alternative. The popularity of the former methods is partly due to the ignorance of programmers to its performance implications. I have seen huge code bases that had to be retired because of their over reliance on OO, and dynamic dispatch.

> I have seen huge code bases that had to be retired because of their over reliance on OO, and dynamic dispatch.

What were they replaced with?

Re: The cost of dynamic vs. static dispatch in C++ (2013)

#27

Post by kazinator » Sun, Mar 01, 2015, 1:38 AM UTC

Rusky wrote: post #18

bit_razor wrote: post #11

How is this an apples to apples comparison? Instantiating CRTPInterface with Implementation is no different than just calling DynamicImplementation::tick() directly, so why not benchmark that?

The entire point of the benchmark is virtual calls vs templated static calls. Everything else is what should be controlled.

If everything else can be controlled, then virtual calls are being used unnecessarily. The fact that an indirect calling mechanism which has to first use indirection to retrieve the address to be called (and possibly do more work, like fixing up a pointer) is slower than a static call is completely unsurprising.

If virtual calls are used essentially, then it's a canoe-versus-bicycle comparison, because everything else cannot be controlled. The program based on static calls has to be written quite differently to solve the same problem, and the benchmark then measures the entire approach. Plus the benchmark doesn't account for benefits that it doesn't measure, like maintainability and extensibility of the code.

Re: The cost of dynamic vs. static dispatch in C++ (2013)

#28

Post by eru » Sun, Mar 01, 2015, 2:29 AM UTC

Jimmy wrote: post #23

stinos wrote: post #13

Earlier quoted context omitted.

If C++ will exist in 10 years from now Apart from it being quite an interesting do-it-all type of language, solely seeing the huge amounts of existing codebases in 'slow' fields like production/manufacturing industries I am pretty sure it will.

Even if new applications stopped being written in C++ today, the amount of legacy code written in C++ would ensure that there would be jobs for C++ programmers for many decades to come. But of course, people will almost certainly still be writing new C++ code 10 years from now.

Just like they are still writing new COBOL.

Re: The cost of dynamic vs. static dispatch in C++ (2013)

#29

Post by pjmlp » Sun, Mar 01, 2015, 7:44 AM UTC

eru wrote: post #28

Jimmy wrote: post #23

Earlier quoted context omitted.

Even if new applications stopped being written in C++ today, the amount of legacy code written in C++ would ensure that there would be jobs for C++ programmers for many decades to come. But of course, people will almost certainly still be writing new C++ code 10 years from now.

Just like they are still writing new COBOL.

And earning lots of money in doing so.

Re: The cost of dynamic vs. static dispatch in C++ (2013)

#30

Post by Rusky » Sun, Mar 01, 2015, 8:11 AM UTC

kazinator wrote: post #27

Rusky wrote: post #18

Earlier quoted context omitted.

The entire point of the benchmark is virtual calls vs templated static calls. Everything else is what should be controlled.

If everything else can be controlled, then virtual calls are being used unnecessarily. The fact that an indirect calling mechanism which has to first use indirection to retrieve the address to be called (and possibly do more work, like fixing up a pointer) is slower than a static call is completely unsurprising. If virtual calls are used essentially, then it's a canoe-versus-bicycle comparison, because everything els…

Regardless, it's still useful to actually test the performance of virtual vs static calls- how much slower are they? in which situations can the compiler devirtualize them?

Or should we only ever benchmark the entire software stack at once, even though people do in fact sometimes use CRTP and virtual functions in the same situations?