Earlier quoted context omitted.
Unless your amalgamated source file is getting into the 100s of megabytes to gigabytes range, I doubt that would be an issue.
If template instantiations use up AST node identifiers (I don't know whether whether they do), I can see scenarios where crazy amplified template instantiation chains can eat substantial chunks of that range.
Working with jumbo/unity builds in C/C++
31–40 of 45 posts
Re: Working with jumbo/unity builds in C/C++
#32> It's unfortunate that "Unity Build" is the prevailing term because it's impossible to do a search without getting a lot of results about the Unity game engine. I ignored this article on the front page because of that. Only because it stayed on the front page for several hours did (and because I care about C++ build issues) did I eventually click on it.
> and because I care about C++ build issues Look no further, these builds will give you more than enough issues on any sizable project.
Re: Working with jumbo/unity builds in C/C++
#33It's only faster if you are building from scratch, and even then it might not be faster. The resources required to build one huge compilation unit are also higher than compiling separately. If you support this type of build, it should not be the only option.
Re: Working with jumbo/unity builds in C/C++
#34Unreal Engine does unity builds quite well - it will, as part of a prebuild step: 1. Merge related cpp and h files together in groups into monolith files, usually in the order of 10-20 source files merged based on my observations - often entire modules will be grouped together. 2. Exclude any individual files from the monolith that have edits since the last change. It's a nice middle ground, you don't substantially s…
Unreal's unity builds get very annoying as soon as your teams exceeds a handful of developers. You often have code that has builds fine on a machine but doesn't on another developer - it completely depends on what order the UBT picks. This is even more annoying when you have some people on Windows, some on Linux, some on Macs, and they consistently get different results. The fact that most tools (Intellisense, all cl…
> We worked around this by introducing a mandatory pre-merge CI stage that constantly does non-unity builds - but it's costly and not something a small company can often afford (a non-unity build of our UE project is ~20min on a Linux runner, and way more on a Windows one. That adds up fast).
Or you could only build the files that have changed. Even the largest of large files are one minute compiles.
> Unreal itself hasn't been non-unity buildable for a very long time.
Unreal builds in non-unity just fine. There was definitely a time period where it _didn't_, but for the last few years it's been much better than that.
Re: Working with jumbo/unity builds in C/C++
#35It's only faster if you are building from scratch, and even then it might not be faster. The resources required to build one huge compilation unit are also higher than compiling separately. If you support this type of build, it should not be the only option.
It is always faster in all cases from my experience. It should not, but it is. Try.
Re: Working with jumbo/unity builds in C/C++
#36It's only faster if you are building from scratch, and even then it might not be faster. The resources required to build one huge compilation unit are also higher than compiling separately. If you support this type of build, it should not be the only option.
It is always faster in all cases from my experience. It should not, but it is. Try.
The best possibility is if your build system can detect which files have changed and exclude them from unity builds, as this gives you the "slow path" the first time you change a file, but from then on you get the fast path behaviour.
Re: Working with jumbo/unity builds in C/C++
#37Unfortunately this isn't consistent with how these terms are typically used. This article only explains the simple case where your program is small enough that you can compile all source files at once. That's not realistic or practical for much larger projects. Both WebKit and Chromium support unity/jumbo builds. They combine around 20 source files at a time into a single compilation unit, which provides a reasonable…
For years we rolled our own unity build, but now CMake supports it directly through CMAKE_UNITY_BUILD and CMAKE_UNITY_BUILD_BATCH_SIZE, making it straightforward to enable.
When first enabling it on a large project, you'll run into clashes where different files contain identically-named file-scoped function or variables. Sometimes this reveals copy-pasted code, where the fix is to refactor the duplicate code anyway. Other times you just pick more specific names to avoid the clash.
We find unity build gives a solid 3X build speedup. We haven't eliminated header files, and in fact keep one slow CI job building the code without unity to ensure our code still builds either way.
Re: Working with jumbo/unity builds in C/C++
#38Unreal Engine does unity builds quite well - it will, as part of a prebuild step: 1. Merge related cpp and h files together in groups into monolith files, usually in the order of 10-20 source files merged based on my observations - often entire modules will be grouped together. 2. Exclude any individual files from the monolith that have edits since the last change. It's a nice middle ground, you don't substantially s…
Unreal's unity builds get very annoying as soon as your teams exceeds a handful of developers. You often have code that has builds fine on a machine but doesn't on another developer - it completely depends on what order the UBT picks. This is even more annoying when you have some people on Windows, some on Linux, some on Macs, and they consistently get different results. The fact that most tools (Intellisense, all cl…
Re: Working with jumbo/unity builds in C/C++
#39Unreal Engine does unity builds quite well - it will, as part of a prebuild step: 1. Merge related cpp and h files together in groups into monolith files, usually in the order of 10-20 source files merged based on my observations - often entire modules will be grouped together. 2. Exclude any individual files from the monolith that have edits since the last change. It's a nice middle ground, you don't substantially s…
Re: Working with jumbo/unity builds in C/C++
#40Unfortunately this isn't consistent with how these terms are typically used. This article only explains the simple case where your program is small enough that you can compile all source files at once. That's not realistic or practical for much larger projects. Both WebKit and Chromium support unity/jumbo builds. They combine around 20 source files at a time into a single compilation unit, which provides a reasonable…
Firefox uses “unified” builds, concatenating .cpp files only within each directory. Unified Firefox builds are about 2-5x faster on my Mac. Non-unified builds are still built to make sure there are no unexpected side effects or accidental header file dependencies. https://firefox-source-docs.mozilla.org/build/buildsystem/un...