Distcc – distribute builds across multiple machines simultaneously (2006)
1–10 of 27 posts
Re: Distcc – distribute builds across multiple machines simultaneously (2006)
#2If you like distcc, perhaps you'd also like icecream https://github.com/icecc/icecream which I think is a bit easier to use.
Re: Distcc – distribute builds across multiple machines simultaneously (2006)
#3Is distcc actually any faster than running the builds locally on a half decent machine? If I recall the latency associated with getting the file to the build machines often exceeded the local compilation duration. I remember being really excited about the idea, and really disappointed with the results.
Re: Distcc – distribute builds across multiple machines simultaneously (2006)
#4Is distcc actually any faster than running the builds locally on a half decent machine? If I recall the latency associated with getting the file to the build machines often exceeded the local compilation duration. I remember being really excited about the idea, and really disappointed with the results.
One use that I can think of (never tried it myself, though in theory it should work) is if you are trying to cross compile for a processor architecture that doesn't have decent speed (many embedded chips), and the package you are compiling is difficult to cross compile (where the build script makes decisions based on the local environment or is otherwise to get to play nice with a cross compiler).
In that case you can run the build on the target CPU, but instead of calling gcc locally it calls distcc pointing to a cross compiler installed on a fast machine. This can be useful if you are compiling a bunch of packages from a distribution.
Re: Distcc – distribute builds across multiple machines simultaneously (2006)
#5Is distcc actually any faster than running the builds locally on a half decent machine? If I recall the latency associated with getting the file to the build machines often exceeded the local compilation duration. I remember being really excited about the idea, and really disappointed with the results.
Running it locally will always be faster as long as your machine is not a bottleneck (#cores, ram, ...). I think the use-case for distcc et al is to enable less-powerful machines to run builds faster by levering other machines. That’s exactly what we use it for at work. Our developers have not-so-powerful laptops and with distcc/icecc they can utilize the power of our build agents in the server room.
Also interesting to read: https://github.com/StanfordSNR/gg
Re: Distcc – distribute builds across multiple machines simultaneously (2006)
#6Is distcc actually any faster than running the builds locally on a half decent machine? If I recall the latency associated with getting the file to the build machines often exceeded the local compilation duration. I remember being really excited about the idea, and really disappointed with the results.
Running it locally will always be faster as long as your machine is not a bottleneck (#cores, ram, ...). I think the use-case for distcc et al is to enable less-powerful machines to run builds faster by levering other machines. That’s exactly what we use it for at work. Our developers have not-so-powerful laptops and with distcc/icecc they can utilize the power of our build agents in the server room. Also interesting…
Or maybe if money is no object spin up a bunch of VMs in the cloud to compile.
Re: Distcc – distribute builds across multiple machines simultaneously (2006)
#7Just use Bazel with remote execution. It can distribute your unit tests, too.
Re: Distcc – distribute builds across multiple machines simultaneously (2006)
#8Is distcc actually any faster than running the builds locally on a half decent machine? If I recall the latency associated with getting the file to the build machines often exceeded the local compilation duration. I remember being really excited about the idea, and really disappointed with the results.
We used to use distcc (2007 time period) for the daily builds of one of our large, in-house C++ products - order of 10M LOC.
In principle, it was a good use-case with a highly modular structure and a clearly defined but chunky build graph.
In practice, it did work, but throwing more hardware at the problem on a single host turned out to be faster than the existing distcc setup and had much reduced operational complexity.
We could probably have tuned it further with distcc plus the new hardware, but we achieved the performance target we were looking for.
Re: Distcc – distribute builds across multiple machines simultaneously (2006)
#9Does this cache build artifacts in any way? So if several developers are working on the same code-base, they can share build artifacts based on, like, the hash of the inputs? That would require reproducible builds, but that's probably not too tricky to achieve.
Re: Distcc – distribute builds across multiple machines simultaneously (2006)
#10Does this cache build artifacts in any way? So if several developers are working on the same code-base, they can share build artifacts based on, like, the hash of the inputs? That would require reproducible builds, but that's probably not too tricky to achieve.
It doesn’t IIRC. But, you can combine it with ccache which can use a NFS share.
If you use clang, that might be of interest: https://github.com/yrnkrn/zapcc