anything similar for higher level languages (c# or the likes)?
I think there is usually something else to optimize before this becomes a problem. And if it becomes a problem you need a low-level language anyway?
The cost of dynamic vs. static dispatch in C++
21–30 of 72 posts
Re: The cost of dynamic vs. static dispatch in C++
#22anything similar for higher level languages (c# or the likes)?
The Java Hotspot VM can still optimize for this case, if the virtual call leads to only a few classes most of the time. Several virtual methods can be inlined, but of course there's still an extra step compared to static dispatch: the classes of the current object and the inlined methods have to be compared. If no matching method is inlined, control needs to be passed back to the VM.
Re: The cost of dynamic vs. static dispatch in C++
#23for (unsigned i = 0; i tick(j); } } I wouldn't go quite so far as to say that benchmarks with tight inner loops like this are completely useless, but they are nearly so. The author is clearly aware that the real world of performance is much bigger & more complex than his simple Petri dish. Credit to him for mentioning that. It's also really refreshing to see him analysing the optimised assembly. The trouble with this…
And I have seen projects whose performance was crippled by layers upon layers of endless virtual calls. YMMV ;-)
Agreed, for almost all code it doesn't matter, but for the remaining small fraction it's worth thinking about these things. It sounds pretty insane to go with a blanket approach of removing virtual calls throughout an entire codebase without understanding which ones are the problematic ones. Especially since some ways of solving the problem could potentially lead to other problems like increased compiled code size.
I've seen plenty of software (especially systems software) that does spend much of it's time in tight inner loops. Pulling out all the optimization stops there can give measurable gains. I've personally seen measurable gains on real applications from tricks like reordering branches so that the more predictable branches go first.
Re: The cost of dynamic vs. static dispatch in C++
#24its interesting to see a break down of this - especially using modern compilers on the intel platform. did you try the intel compiler? for raw low level optimisation it sometimes massively out performs the ms, gcc or clang versions... i'd imagine these problems are worse on ARM chips, and dynamic dispatch is even less effective there - certainly on PPC architectures I've seen much worse performance than on similarly…
You may be thinking of this: IIRC the standard recommends that compilers omit dynamic dispatch when the dynamic type is known at compile time - this essentially boils down to the case where a virtual method call follows creation of the object with 'new' or as an automatic variable. In my experience, this is commonly implemented correctly in compilers.
The other case where the dynamic type is known is in the constructor itself of course.
Re: The cost of dynamic vs. static dispatch in C++
#25I'd like to see a comparison of calling a dynamically linked function call vs a non-dynamically linked virtual call. Dynamic linking has more indirection than you might expect because the function addresses can't always just be put at the call site during the library load (the places where you would want to write the address can be in code that is read-only mmapped to aid in sharing memory between processes and to av…
The calls into jump tables are generally static, so the jump table itself can be prefetched. The jump table code is then a regular function pointer call, which is also monomorphic and so can be reliably predicted. I'd expect the impact to be small compared to a regular monomorphic function pointer call.
Re: The cost of dynamic vs. static dispatch in C++
#26anything similar for higher level languages (c# or the likes)?
This is less of a problem for systems with JIT compilation (including Java, C#, etc). They can recompile the code at runtime, which allows some nice tricks for virtual calls. They can turn a virtual call into a regular call with inline caching (http://en.wikipedia.org/wiki/Inline_caching), or can even compile a specialized version of the code for a given type and inline the entire virtual function.
Re: The cost of dynamic vs. static dispatch in C++
#27anything similar for higher level languages (c# or the likes)?
The Java Hotspot VM can still optimize for this case, if the virtual call leads to only a few classes most of the time. Several virtual methods can be inlined, but of course there's still an extra step compared to static dispatch: the classes of the current object and the inlined methods have to be compared. If no matching method is inlined, control needs to be passed back to the VM.
A fascinating article on this type of optimisation:
http://www.azulsystems.com/blog/cliff/2010-04-08-inline-cach...
Re: The cost of dynamic vs. static dispatch in C++
#28for (unsigned i = 0; i tick(j); } } I wouldn't go quite so far as to say that benchmarks with tight inner loops like this are completely useless, but they are nearly so. The author is clearly aware that the real world of performance is much bigger & more complex than his simple Petri dish. Credit to him for mentioning that. It's also really refreshing to see him analysing the optimised assembly. The trouble with this…
And I have seen projects whose performance was crippled by layers upon layers of endless virtual calls. YMMV ;-)
Yes, of course. But the real challenge is to determine that it's your virtual indirection that's causing the problem. Once you know what's causing the problem, fixing it is (relatively) easy.
The danger is that benchmarks like this encourage naïve programmers to use complex constructs as a matter of course, when simpler would usually be better. "Premature optimisation is the root of all evil" and all that...
Re: The cost of dynamic vs. static dispatch in C++
#29anything similar for higher level languages (c# or the likes)?
The .NET and Java VMs are in fact able to do some inlining on virtual/interface calls and do other sorts of smart dispatch. So the cost of a virtual method in .NET and Java is not necessarily equivalent to the cost in C++.
Re: The cost of dynamic vs. static dispatch in C++
#30So he found that dynamic dispatch was a lot more expensive. Fair enough and not very surprising. But let's quantify it a bit in absolute terms. The dynamic version of the code took 1.25s to run, during which time it performed approximately 8 x 10^8 virtual function calls. That translates to a cost per call of 1.5 nanoseconds.
From which my takeaway would be: In inner-loopy code for which an extra nanosecond or so per call is critical, you should avoid virtual function calls. For anything else, don't worry about it.