11–20 of 45 posts
One issue that can happen that the article didn't mention is running out of AST node identifiers in Clang. Clang uses a 32-bit number to identify AST nodes. If a single translation unit is large enough, it can overflow this and you can get some very weird compilation errors.
IMO separation of interface and implementation is one of the "good" things about C/C++ (I used to be confused about it when starting out), it gives you a good overview of a piece of code and how it is supposed to be used. In other languages with no such "feature", you'll have to scroll hundreds of lines of implementation details you don't care about to understand the interface. You say this as a possible solution: >…
My experience was that there was decent benefit to keeping the .h/.cpp build running. Most 'normal' C++ tools and IDEs are not going to assume you are using a Unity build and tend to choke on it. Even though we never really shipped anything from the non-Unity build, having it around was useful for avoiding 'phantom' errors in the IDE and having static analysis tools work properly.
Unfortunately this isn't consistent with how these terms are typically used. This article only explains the simple case where your program is small enough that you can compile all source files at once. That's not realistic or practical for much larger projects. Both WebKit and Chromium support unity/jumbo builds. They combine around 20 source files at a time into a single compilation unit, which provides a reasonable…
Firefox uses “unified” builds, concatenating .cpp files only within each directory. Unified Firefox builds are about 2-5x faster on my Mac. Non-unified builds are still built to make sure there are no unexpected side effects or accidental header file dependencies. https://firefox-source-docs.mozilla.org/build/buildsystem/un...
https://news.ycombinator.com/item?id=35825683 - Unity builds lurked into the Firefox Build System (2023)
The page that was referenced by that thread has moved, current location is
https://serge-sans-paille.github.io/pythran-stories/how-unit...
1. Merge related cpp and h files together in groups into monolith files, usually in the order of 10-20 source files merged based on my observations - often entire modules will be grouped together. 2. Exclude any individual files from the monolith that have edits since the last change.
It's a nice middle ground, you don't substantially slow down your incremental builds since the files you're editing are still compiled individually, plus you get a fairly substantial improvement in build times.
It's nice in that you can still structure your source in separate cpp/h files will little extra consideration for the unity builds and it mostly "just works" in both unity mode and with regular builds.
Unfortunately you occasionally do have issues with builds that work in unity but not in individual compilation or vice versa (usually due to arcane #include dependency chains) but they're usually easy to fix.
Unreal Engine does unity builds quite well - it will, as part of a prebuild step: 1. Merge related cpp and h files together in groups into monolith files, usually in the order of 10-20 source files merged based on my observations - often entire modules will be grouped together. 2. Exclude any individual files from the monolith that have edits since the last change. It's a nice middle ground, you don't substantially s…
We worked around this by introducing a mandatory pre-merge CI stage that constantly does non-unity builds - but it's costly and not something a small company can often afford (a non-unity build of our UE project is ~20min on a Linux runner, and way more on a Windows one. That adds up fast).
Unreal itself hasn't been non-unity buildable for a very long time. In general IMHO Unity builds are a testament to the failure of the C++ standards committee to realise that modules and the building model should be part of the standard to. The "one file is a translation unity" hasn't been adequate for years IMHO - I honestly appreciate how Rust basically imposed cargo as a standard, it was a hard but sane choice.
One issue that can happen that the article didn't mention is running out of AST node identifiers in Clang. Clang uses a 32-bit number to identify AST nodes. If a single translation unit is large enough, it can overflow this and you can get some very weird compilation errors.
Unless your amalgamated source file is getting into the 100s of megabytes to gigabytes range, I doubt that would be an issue.
One issue that can happen that the article didn't mention is running out of AST node identifiers in Clang. Clang uses a 32-bit number to identify AST nodes. If a single translation unit is large enough, it can overflow this and you can get some very weird compilation errors.
Unless your amalgamated source file is getting into the 100s of megabytes to gigabytes range, I doubt that would be an issue.
Unreal Engine does unity builds quite well - it will, as part of a prebuild step: 1. Merge related cpp and h files together in groups into monolith files, usually in the order of 10-20 source files merged based on my observations - often entire modules will be grouped together. 2. Exclude any individual files from the monolith that have edits since the last change. It's a nice middle ground, you don't substantially s…
Unreal's unity builds get very annoying as soon as your teams exceeds a handful of developers. You often have code that has builds fine on a machine but doesn't on another developer - it completely depends on what order the UBT picks. This is even more annoying when you have some people on Windows, some on Linux, some on Macs, and they consistently get different results. The fact that most tools (Intellisense, all cl…
We target windows, linux, xsx and ps5 and have probably 15-20 programmers making contributions daily and probably only hit maybe one or two of these issues per week, and they rarely get checked in as we get them during our mandatory preflight build during code review, similar to what you describe. We run all the preflight on on-prem machines now so the cost is minimized compared to our former cloud solution.
We did a lot of work to modularize our codebase so maybe that is helping?
Combing many cpp files into a single translation unit is a good idea. More projects should do this. Infact I’d go so far as to say that most non-trivial, popularC++ projects on GitHub could and should probably be boiled down to a single translation unit.
The amount of redundant compiling in C++ is insane. Any project that uses STL is compiling the same crap over and over and over and over and over. Then counting on the linker to deduplicate the billions of cycles of wasted work.