Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

C++ Headers are Expensive

virtuallyrandom.com

Post reply on HN
Print view View original

41–50 of 115 posts

Re: C++ Headers are Expensive

#41

Post by kllrnohj »

nanolith wrote: post #18
shereadsthenews wrote: post #16

Earlier quoted context omitted.

That only makes sense if you are planning to offer two implementations of your library. Which I of course urge you to not do. This article is about the STL headers. The reason std::sort beats the pants off all other languages’ sort routines is because the iterators of every collection, all specializations of swap, and the comparator can all be visible to the compiler. If they weren’t, it would be a lot slower. Premat…

> Premature optimization is not really a thing Okay. I'm going to stop this thread right there and take some opportunity to provide some mentoring. I hope you accept this, as it will help in your career. Read this paper. It is a classic. https://pic.plover.com/knuth-GOTO.pdf

You should read the paper a bit more closely.

"We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3 %. A good programmer will not be lulled into complacency by such reasoning, he will be wise to look carefully at the critical code; but only after that code has been identified"

We know that cache misses are not a small in-efficiency. This has been measured & observed on many real systems as a real, systemic problem. It's why data-oriented design is currently the King of game engine world, because cache misses kill performance. It is not premature to warn against it as a general practice as a result, as that's systemic de-optimization that will likely impact the critical 3%.

Re: C++ Headers are Expensive

#42

Post by kllrnohj »

cgrealy wrote: post #36
nanolith wrote: post #29

Earlier quoted context omitted.

If someone says that premature optimization isn't a thing, I don't think it is condescending to point out that it is by posting original source material. :-)

> it only wastes the programmer’s time. ... Wasting CPU time on the other hand bothers me a lot! As always the answer is... it depends. Programmer time costs money, CPU time is cheap by comparison. If you're building something that runs occasionally, or is IO/UI/network bound... CPU time is largely irrelevant. But if you're building something that runs in a tight loop or a library that will be compiled in millions of…

The pImpl pattern costs programmer time as it's more code to write. By contrast compiling is just CPU time and you can trivially throw a bigger workstation at the problem.

Re: C++ Headers are Expensive

#43

Post by branchless »

kccqzy wrote: post #24
shereadsthenews wrote: post #9

Earlier quoted context omitted.

pImpl pattern is great for those who don’t care about performance but it’s inappropriate for most header libraries. You wouldn’t want a library that hides the implementation of std::vector for example. With a visible implementation the compiler compile e.g. operator[] down one x86 instruction. With a pImpl pattern it will be an indirect function call in all likelihood that will be hundreds of times slower. It can mak…

Using the pimpl pattern doesn't mean an indirect function call. The function to be called is always known. It's just an extra indirection in the data member. It's cheap. Think of it as Java style memory layout: everything that's not primitive stored in an object is a reference and therefore behind one level of indirection. The performance of Java is acceptance in the vast majority of use cases. Using pimpl will be th…

The only advantage of c++ is max perf. If we could skip a beat we couldn't justify using c++.

Re: C++ Headers are Expensive

#44

Post by kazinator »

Speaking about GNU C++ (and C), the headers are getting cheaper all the time compared to the brutally slow compilation.

Recently, after a ten year absence of not using ccache, I was playing with it again.

The speed-up from ccache you obtain today is quite a bit more more than a decade ago; I was amazed.

ccache does not cache the result of preprocessing. Each time you build an object, ccache passes it through the preprocessor to obtain the token-level translation unit which is then hashed to see if there is a hit (ready made .o file can be retrieved) or miss (preprocessed translation unit can be compiled).

There is now more than a 10 fold difference between preprocessing, hashing and retrieving a .o file from the cache, versus doing the compile job. I just did a timing on one program: 750 milliseconds to rebuild with ccache (so everything is preprocessed and ready-made .o files are pulled out and linked). Without ccache 18.2 seconds. 24X difference! So approximately speaking, preprocessing is less than 1/24th of the cost.

Ancient wisdom about C used to be that more than 50% of the compilation time is spent on preprocessing. That's the environment from which came the motivations for devices like precompiled headers, #pragma once and having compilers recognize the #ifndef HEADER_H trick to avoid reading files.

Nowadays, those things hardly matter.

Nowdays when you're building code, the rate at which .o's "pop out" of the build subjectively appears no faster than two decades ago, even though the memories, L1 and L2 cache sizes, CPU clock speeds, and disk spaces are vastly greater. Since not a lot of development has gone into preprocessing, it has more or less sped up with the hardware, but overall compilation hasn't.

Some of that compilation laggardness is probably due to the fact that some of the algorithms have tough asymptotic complexity. Just extending the scope of some of the algorithms to do a bit of better job causes the time to rise dramatically. However, even compiling with -O0 (optimization off), though faster, is still shockingly slow, given the hardware. If I build that 18.2 second program with -O2, it still takes 6 seconds: an 8X difference compared to preprocessing and linking cached .o files in 750 ms. A far cry from the ancient wisdom that character and token level processing of the source dominates the compile time.

Re: C++ Headers are Expensive

#45

Post by nanolith »

kllrnohj wrote: post #41
nanolith wrote: post #18

Earlier quoted context omitted.

> Premature optimization is not really a thing Okay. I'm going to stop this thread right there and take some opportunity to provide some mentoring. I hope you accept this, as it will help in your career. Read this paper. It is a classic. https://pic.plover.com/knuth-GOTO.pdf

You should read the paper a bit more closely. "We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3 %. A good programmer will not be lulled into complacency by such reasoning, he will be wise to look carefully at the critical code; but only after that code has been identified" We know that c…

I think you may want to read the quote from the paper a bit more carefully. "...he will be wise to look carefully at the critical code; but only after that code has been identified"

I was told that "premature optimization is not really a thing" as a response to a reply I received that pImpls should be avoided at all costs.

When we analyze the performance impact of software, we don't shotgun change things because of a generalized fear of cache misses. We examine critical code paths and make changes there based on profile feedback. That is the spirit of what Knuth is saying in this quote. Look carefully at critical code, BUT ONLY AFTER that code has been identified.

A cache miss is critical when it is in a critical path. So, we write interfaces with this in mind. Compilation time matters, as does runtime performance. Either way, we identify performance bottlenecks as they come up and we optimize them. Avoiding clearer coding style, such as encapsulation, because it MIGHT create faster code, is counter-productive.

We can apply the Pareto Principle to understand that 80% of the performance overhead can be found in 20% of the code. The remaining 80% of the code can use pImpls or could be rewritten in Haskell for all it matters for performance. But, that code still needs to be compiled, and often by clients who only have headers and library files. Subjecting them to long compiles to gain an unimportant improvement in performance in code that only rarely gets called is a bad trade. Spend that time optimizing the parts of the code that matter, which, as Knuth says, should only be done after this code has been identified.

EDIT: the downvoting on this comment is amusing, given that "avoid pImpls" is exactly the sort of 97% cruft that Knuth was addressing.

Re: C++ Headers are Expensive

#46

Post by [deleted] »

nanolith wrote: post #18
shereadsthenews wrote: post #16

Earlier quoted context omitted.

That only makes sense if you are planning to offer two implementations of your library. Which I of course urge you to not do. This article is about the STL headers. The reason std::sort beats the pants off all other languages’ sort routines is because the iterators of every collection, all specializations of swap, and the comparator can all be visible to the compiler. If they weren’t, it would be a lot slower. Premat…

> Premature optimization is not really a thing Okay. I'm going to stop this thread right there and take some opportunity to provide some mentoring. I hope you accept this, as it will help in your career. Read this paper. It is a classic. https://pic.plover.com/knuth-GOTO.pdf

[deleted]

Re: C++ Headers are Expensive

#47

Post by nanolith »

kllrnohj wrote: post #42
cgrealy wrote: post #36

Earlier quoted context omitted.

> it only wastes the programmer’s time. ... Wasting CPU time on the other hand bothers me a lot! As always the answer is... it depends. Programmer time costs money, CPU time is cheap by comparison. If you're building something that runs occasionally, or is IO/UI/network bound... CPU time is largely irrelevant. But if you're building something that runs in a tight loop or a library that will be compiled in millions of…

The pImpl pattern costs programmer time as it's more code to write. By contrast compiling is just CPU time and you can trivially throw a bigger workstation at the problem.

This depends on who is writing the code and who is compiling the headers. A software developer who is building headers for someone else (an internal or external client) may trade the overhead of this pattern for a faster compilation time. Reducing compilation time by 80% may be well worth the overhead of adding 10% more code to an interface.

It is not always possible to just throw more hardware at the problem of compilation. For instance, one may be using a build pipeline that requires specific steps to be followed as part of gating tasks. The time it takes to compile code over and over again for unit testing, behavioral testing, acceptance testing, integration testing, etc., each impacts delivery time and handoff.

Earlier in my career, I worked with a code base that was approximately 10 million lines of code in size. Compiling this code base would take approximately 7 hours on the best hardware we could buy. The C++ developers were adamant about ensuring that their headers were "complete" as they called it. With a few changes, such as forward declarations, abstract interfaces, and encapsulation, my team was able to reduce that compile time to less than 35 minutes. Based on profile feedback, we saw less than a quarter of a percent difference in overhead. Productivity-wise, we managed to reduce developer workflows so significantly that it was a better use of our time for overall cost than working on a billable project.

Most projects aren't nearly that bad, but it does go to show that it is possible to significantly reduce compilation time without significantly impacting runtime performance, even in C++.

Re: C++ Headers are Expensive

#48

Post by Silhouette »

nanolith wrote: post #34
nikanj wrote: post #22

Earlier quoted context omitted.

Knuth wasn't saying "just ignore performance altogether", he was saying "stop making things needlessly complicated for the last bit of juice".

For instance, pulling in the STL for an interface header instead of encapsulating these details. :-) No one is claiming that we should ignore performance all together. But, understanding through profiling where performance issues are and designing toward a faster implementation is more important than trying to inline definitions up front.

Do you also write all your functions to pass large inputs by value until a profiler says you can pass a const reference?

Some implementation details are so well understood that you really don't need a profiler to do what is probably the right thing by default.

Re: C++ Headers are Expensive

#49

Post by dkersten »

beached_whale wrote: post #21
nanolith wrote: post #13

Earlier quoted context omitted.

This is true. Fortunately, these do not need to be inlined, which can still free client code of compile time overhead. It's a tradeoff between compile time and complexity.

Another option is something like pimp but keep the state in the public class. Now you get stack allocation but the private part is still private and firewalling all the other headers and details not needed for the public interface. Just pass this to the private methods. Edit: Something like https://github.com/beached/stack_pimpl

I don’t get the example, can you explain? What does this buy that you don’t have by simply putting the content of private.cpp into stack_pimpl.cpp? The data members are already in stack_pimpl.h (so nothing private or hidden about that) and the methods are already declared in both and implmented in both cpp files, so what are you buying over just putting the declarations in one header and implementation in one source and not delegating from one to the other?

Was it just a oversimplistic example and the benefit is actually if priv_t has a bunch of internal methods that you want to keep out of the stack_pimpl.h interface?

Re: C++ Headers are Expensive

#50

Post by repsilat »

Someone wrote: post #31
kccqzy wrote: post #24

Earlier quoted context omitted.

Using the pimpl pattern doesn't mean an indirect function call. The function to be called is always known. It's just an extra indirection in the data member. It's cheap. Think of it as Java style memory layout: everything that's not primitive stored in an object is a reference and therefore behind one level of indirection. The performance of Java is acceptance in the vast majority of use cases. Using pimpl will be th…

”It's just an extra indirection in the data member. It's cheap” That extra indirection often means a cache miss. That isn’t cheap. Accessing each item traversed through a pointer can easily halve program speed. Java tries hard to prevent the indirections (local objects may live in the stack, their memory layout need not follow what the source code say, objects may even only exist in cpu registers)

Hmm... if you were a horrible person you could declare a `char[n]` member instead of a pointer. Then you could placement-new the impl in the constructor, and static-assert that `sizeof(impl)>=n`... No more cache misses :-).

:-(

Post reply on HN