Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unity builds lurked into the Firefox Build System

serge-sans-paille.github.io

Post reply on HN
Print view View original

31–40 of 64 posts

Re: Unity builds lurked into the Firefox Build System

#31

Post by robalni »

simplotek wrote: post #26
robalni wrote: post #22

I always use unity builds for all my projects now. That combined with using tcc as compiler (for C code) makes builds really fast. Another nice feature of unity builds is that I don't need to declare functions twice and keep the declarations synced. It's also nice to only have one place to find information about a function; people often put comments in header files that you can miss if you go to the definition. All o…

> Another nice feature of unity builds is that I don't need to declare functions twice and keep the declarations synced. What exactly leads you to have multiple declarations in sync, and thus creating the to "keep [multiple] declarations synced"?

I mean if you use multiple translation units and header files, you need to have a copy of the function declaration in that header file to be able to call it from other translation units.

Re: Unity builds lurked into the Firefox Build System

#32

Post by pjmlp »

thinkling wrote: post #19

I’ve been out of C/C++ development for a long time but seem to remember that precompiled headers were a thing back in the day. That approach didn’t have the name space issues pointed out here. Why are precompiled headers not used anymore?

As far as I am aware, they never work that great on UNIX compilers, as no big effort was ever spent improving them.

About 20 years ago, on UNIX workloads we used to speed the compilation via ClearMake, a kind of distributed version of code cache that would plug into the compilers, however it has part of ClearCase SCM product.

On Windows, with Microsoft and Borland (nowadays Embarcadero), they work quite alright.

Also, modules will fix that, as per VC++ reports, importing the whole standard library (import std, as per C++23) takes a fraction of only including iostream.

Re: Unity builds lurked into the Firefox Build System

#33

Post by maccard »

thinkling wrote: post #19

I’ve been out of C/C++ development for a long time but seem to remember that precompiled headers were a thing back in the day. That approach didn’t have the name space issues pointed out here. Why are precompiled headers not used anymore?

They're very much alive and well on MSVC. Our work projects use both unity builds _and_ precompiled headers.

Re: Unity builds lurked into the Firefox Build System

#34

Post by maccard »

simplotek wrote: post #30
mastax wrote: post #14

Interesting. I've been aware of this technique for years because of the SQLite Amalgamation, but that was always sold as a way to simplify distribution and perhaps improve performance of the binary. I hadn't considered it as a build speed optimization, though that seems somewhat obvious in hindsight.

> I hadn't considered it as a build speed optimization, though that seems somewhat obvious in hindsight. Some build systems like cmake already support unity builds, as this is a popular strategy to speed up builds. Nevertheless, if speed is the main concern them it's preferable to just use a build cache like ccache, and modularize a project appropriately.

Why not both?

Also, does ccache work with MSVC?

Re: Unity builds lurked into the Firefox Build System

#36

Post by cpeterso »

stephc_int13 wrote: post #28
omoikane wrote: post #17

> This generally leads to faster compilation time in part because it aggregates the cost of parsing the same headers over and over. But this also reduces the opportunity to parallelize compilation across multiple files because they have been concatenated into fewer build units, and each unit now requires more memory to deal with the non-header parts. For some build systems and repositories, this actually increases bu…

On very large projects you can always cut them into several libraries, and compile them on different cores. Quite easy to do in practice.

I believe Firefox builds only unify files within the same directory and a maximum of a ~dozen cpp files per unit. So there are still plenty of build parallelism across directories.

Re: Unity builds lurked into the Firefox Build System

#38

Post by leni536 »

Unity builds mean that you can no longer use internal linkage safely anymore, and that's not something I like to give up. It forces the codebase to follow a certain style that I don't like. Hopefully modules will give the advantage of unity builds without this downside.

Re: Unity builds lurked into the Firefox Build System

#39

Post by 10000truths »

With the advent of LTO, unity builds are mostly a band-aid for poor management of header files. The Linux kernel project was able to net a ~40% reduction in compilation CPU-time just by pruning the contents of some key header files [1].

It really boils down to two rules:

1. Don't declare anything in header files that is only used in one compilation unit. Internal structs and functions should be declared and defined in source files, and internal linkage used wherever possible. gcc and clang's -fvisibility=hidden is useful here.

2. The more frequently a header file is included (whether transitively or directly), the more it should be split up. If a "common" or "utility" header file is included in 10000 source files, then any struct, function, etc. that you add to that file will have to be parsed 10000 times by the compiler every time you build from scratch, even if only 10 source files actually use the struct/function that you added. gcc and clang's -H flag is useful here.

[1] https://lore.kernel.org/lkml/YdIfz+LMewetSaEB@gmail.com/

Re: Unity builds lurked into the Firefox Build System

#40

Post by Scubabear68 »

Headers and C style macros are probably the most unfortunate aspects of C (and by extension, C++).

So many hacks in compilers to try to work around this. A shame there is no language level fix for this nonsense.

Really wish there could be a C++—- that would improve on C in areas like this, and avoid all the incredible nonsense of C++. And no, not Rust or Go.

Post reply on HN