Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Distcc: A fast, free distributed C/C++ compiler

distcc.org

Post reply on HN
Print view View original

31–40 of 103 posts

Re: Distcc: A fast, free distributed C/C++ compiler

#31

Post by pjc50 »

iveqy wrote: post #28

I'm curious about the security implications with using distcc. Doesn't this mean that if one computer gets compromised, the attacker can run code on all other computers using distcc, or secretly inject malicious code in the build result. So using distcc means that all computers using it must be trusted. And that means that using it on "all developers computers to share the load" is good for performance but bad for se…

Everything on the same LAN should generally be treated as "compromised/not compromised" together. There's rarely just a compromise of one machine in the same way there's never just one cockroach.

I'm not sure whether distcc affects reproducible builds?

You could, in any case, have tighter controls on the release builds, which would be done on a CI machine before signing.

(Back when I used distcc we didn't distribute across the dev machines, we had an entire build farm of two racks of 2U servers!)

Re: Distcc: A fast, free distributed C/C++ compiler

#32

Post by donaldihunter »

anonymousDan wrote: post #29

Can anyone explain the architecture/how it works at a high level? I get that it is distributed. Does it basically copy the complete source tree to every worker and have them compile some independent subset of the object files? Does performance scale linearly with the number of worker nodes?

Per the description:

> distcc sends the complete preprocessed source code across the network for each job, so all it requires of the volunteer machines is that they be running the distccd daemon, and that they have an appropriate compiler installed.

So all the "environment" is on the source machine and just a bare compiler is required on the remote machines for compilation.

Re: Distcc: A fast, free distributed C/C++ compiler

#33

Post by pjc50 »

anonymousDan wrote: post #29

Can anyone explain the architecture/how it works at a high level? I get that it is distributed. Does it basically copy the complete source tree to every worker and have them compile some independent subset of the object files? Does performance scale linearly with the number of worker nodes?

Last time I looked, it basically ran per-file "cc -E" on the source machine to get a compilation unit (optionally checking for a ccache cache hit at this point), then piped the result to "cc" running on the target machine, and copied the resulting object file back.

> Does performance scale linearly with the number of worker nodes

Yes, for small N.

Overall scaling was limited by how much "make -j" the source machine could cope with.

Re: Distcc: A fast, free distributed C/C++ compiler

#34

Post by DrBazza »

pjc50 wrote: post #15
ironbound wrote: post #7

Is there a reason to uses this over something like bazel build?

Is bazel build distributed? distcc lets you use a lot of machines to speed up the wallclock-time of your compilation.

It can be. By default it is local.

But it has protobufs interfaces (IIRC), so a distributed build farm would generate the grpc endpoints for their implementation and then you tell bazel on the command line (or via .bazelrc) the address of the build farm it can use.

There's a couple of projects that implement the distributed/grpc part, the main one is https://github.com/bazelbuild/bazel-buildfarm

Re: Distcc: A fast, free distributed C/C++ compiler

#35

Post by vkoskiv »

squarefoot wrote: post #27

Good memories! I had fun with distcc by compiling kernels across a few local machines back when desktops for mere mortals were dog slow, and it helped a lot. I never used it for cross compiling though, which is something that could help today when starting compilations from small embedded boards. Did anyone have success in a mixed environment, such as a small ARM board with native GCC plus one or more faster x86 mach…

Just tried this. I have a slow ARM laptop and a fast x86 desktop. The ARM laptop seems to detect the desktop as specified in the environment variable, but it doesn't accelerate compiling at all. The x86 box sits idle while the ARM laptop compiles. I have the aarch64 cross compiler installed on the desktop as well. Maybe I'm just bad at RTFM, but I can't seem to get it to work.

Re: Distcc: A fast, free distributed C/C++ compiler

#36

Post by Someone »

anonymousDan wrote: post #29

Can anyone explain the architecture/how it works at a high level? I get that it is distributed. Does it basically copy the complete source tree to every worker and have them compile some independent subset of the object files? Does performance scale linearly with the number of worker nodes?

It runs the preprocessor locally, then sends that out to a volunteer node. https://www.distcc.org/distcc-lca-2004.html:

“The client is invoked as a wrapper around the compiler by Make. Because distcc is invoked in place of gcc, it needs to understand every pattern of command line invocations. If the arguments are such that the compilation can be run remotely, distcc forms two new sets of arguments, one to run the preprocessor locally, and one to run the. compiler remotely. If the arguments are not understood by distcc, it takes the safe default of running the command locally. Options that read or write additional local files such assembly listings or profiler tables are run locally”

Scalability:

“Reports from users indicate, distcc is nearly linearly scalable for small numbers of CPUs. Compiling across three identical machines is typically 2.5 to 2.8 times faster than local compilation. Builds across sixteen machines have been reported at over ten times faster than a local builds. These numbers include the overhead of distcc and Make, and the time for non-parallel or non-distributed tasks”

Re: Distcc: A fast, free distributed C/C++ compiler

#37

Post by signa11 »

anonymousDan wrote: post #29

Can anyone explain the architecture/how it works at a high level? I get that it is distributed. Does it basically copy the complete source tree to every worker and have them compile some independent subset of the object files? Does performance scale linearly with the number of worker nodes?

from section 3 (Design) of https://www.distcc.org/distcc-lca-2004.html

'''

distcc distributes work from a client machine to any number of volunteer machines. (Since this is free software, we prefer the term volunteer to the slaves that are used by other systems.)

distcc consists of a client program and a server. The client analyzes the command run, and for jobs that can be distributed it chooses a host, runs the preprocessor, sends the request across the network and reports the results. The server accepts and handles requests containing command lines and source code and responds with object code or error messages.

'''

Re: Distcc: A fast, free distributed C/C++ compiler

#38

Post by nicce »

Erlangen wrote: post #14

Isn't it a misnomer to call it a "compiler"? Even it's github README says otherwise, > distcc is not itself a compiler, but rather a front-end to the GNU C/C++ compiler (gcc), or another compiler of your choice. All the regular gcc options and features work as normal.

I guess it depends. You use this software to convert your source code into the binary blobs efficiently. From high level, it behaves like a compiler, but internals are just using other compilers.

Re: Distcc: A fast, free distributed C/C++ compiler

#39

Post by crote »

choeger wrote: post #13

IIRC, when using distcc you must make sure that the Toolchain is ABI-perfect identified by its path. So: /opt/ourcompany/dev/bin/cc absolutely *needs* to be the same on all machines involved or you risk very hard to spot issues. For that reason, either use the absolute same distribution for everyone (and then run /use/bin/cc but watch out for alternatives!) or roll-out your own toolstack but make sure to put its vers…

Sounds like a great use case for containers, no?

Re: Distcc: A fast, free distributed C/C++ compiler

#40

Post by anthk »

anybodyz wrote: post #5

Fastbuild https://www.fastbuild.org/docs/home.html is the free distributed compilation system many game companies use. The combination of automatic unity builds (simply appending many .cpp source files together into very large combined files), caching and distributed compilation together gives you extremely fast C++ builds. Also supports creating project files that are compatible with XCode and Visual Studio so you c…

Game companies didn't hear of ccache then.
Post reply on HN