Used to use this with Gentoo when doing emerges. Reusing old P3s and stuff, those were the day..
Insert XKCD "Compiling..." meme :)
Distcc: A fast, free distributed C/C++ compiler
21–30 of 103 posts
Re: Distcc: A fast, free distributed C/C++ compiler
#22Back at Uni (two-and-a-half decades ago now), I setup a hacky version of this sort of thing to distribute compilations over several workstations: * use make -j to run multiple task at once * replace¹ gcc with a script that picked a host and ran the task on that² This had many limitations that distcc lists it doesn't have (the machines had to have the same everything installed, the input and output locations for gcc h…
[deleted]
Re: Distcc: A fast, free distributed C/C++ compiler
#23How do these distributed build tools work across the Internet, i.e. outside of a local LAN office setting? Does the increased latency and slower bandwidth become a bottleneck?
If you distribute preprocessed code, you need LAN speeds/latency for it to be faster than a local compilation. Otherwise, you need to have the exact same headers in addition to the exact same compiler installed everywhere. It MAY work otherwise, until it breaks in a subtle / obscure way one day and after tracking it down you will think maybe compilation doesn't take THAT long on a single machine :)
So all in all, I wouldn't say works outside of a well-controlled cluster on a LAN.
Re: Distcc: A fast, free distributed C/C++ compiler
#24Fastbuild https://www.fastbuild.org/docs/home.html is the free distributed compilation system many game companies use. The combination of automatic unity builds (simply appending many .cpp source files together into very large combined files), caching and distributed compilation together gives you extremely fast C++ builds. Also supports creating project files that are compatible with XCode and Visual Studio so you c…
Someone just linked Fastbuild few days ago in one community i follow and i did take a look as i havent heard about that project before. I didnt really like the idea that it would need to have its own build rules but its understandable - where as distcc would just integrate directly to what ever build tool you would be using. Also the syntax didnt look so "pleasing" but i guess the behefits can out weight the learning curve of yet an another build language (compared to meson/bazel/cmake et al) ..
Having distributed builds on msvc thought with free software sounded very promising thought.
Re: Distcc: A fast, free distributed C/C++ compiler
#25Wow, it's still active! I remember when MBP wrote it while we were working at OzLabs together.
One day I'll get back to that rewrite of ccontrol using modern distcc's features...
Re: Distcc: A fast, free distributed C/C++ compiler
#26Re: Distcc: A fast, free distributed C/C++ compiler
#27Good memories! I had fun with distcc by compiling kernels across a few local machines back when desktops for mere mortals were dog slow, and it helped a lot. I never used it for cross compiling though, which is something that could help today when starting compilations from small embedded boards. Did anyone have success in a mixed environment, such as a small ARM board with native GCC plus one or more faster x86 machines with cross compiling tools installed?
Re: Distcc: A fast, free distributed C/C++ compiler
#28I'm curious about the security implications with using distcc. Doesn't this mean that if one computer gets compromised, the attacker can run code on all other computers using distcc, or secretly inject malicious code in the build result.
So using distcc means that all computers using it must be trusted. And that means that using it on "all developers computers to share the load" is good for performance but bad for security.
Re: Distcc: A fast, free distributed C/C++ compiler
#29Can anyone explain the architecture/how it works at a high level? I get that it is distributed. Does it basically copy the complete source tree to every worker and have them compile some independent subset of the object files? Does performance scale linearly with the number of worker nodes?
Re: Distcc: A fast, free distributed C/C++ compiler
#30I used this in a uni lab (good lord 15 years ago) when we needed to compile for a robot. It took maybe an hour or two to compile run directly on the robot but I was able to setup distcc to run on all the lab machines in that room and get things done fast.