41–50 of 64 posts
I always wanted to use mold in the build system I did last year: the metrics are fucking impressive. There were subtle differences to gold and ldd that I didn’t have time to chase down, but it seems like the future.
I can't say I've noticed much of a difference myself, maybe 20% faster at best.
It seems odd to me that we still require .o files at all. Given the final program needs to be storable in RAM + Virtual Memory it surprises me that we still need the intermediate step of pushing to the file system only to then immediately reopen and merge those files. Does someone have more info on this? Or is the reason just legacy? Or is the reason just some ideological “single responsibility” thing?
It seems odd to me that we still require .o files at all. Given the final program needs to be storable in RAM + Virtual Memory it surprises me that we still need the intermediate step of pushing to the file system only to then immediately reopen and merge those files. Does someone have more info on this? Or is the reason just legacy? Or is the reason just some ideological “single responsibility” thing?
It seems odd to me that we still require .o files at all. Given the final program needs to be storable in RAM + Virtual Memory it surprises me that we still need the intermediate step of pushing to the file system only to then immediately reopen and merge those files. Does someone have more info on this? Or is the reason just legacy? Or is the reason just some ideological “single responsibility” thing?
The first thing to point out is that reading a recently-written file isn't all that expensive: it's stored in the filesystem cache anyways, so you bypass the disk for the reads.
Moreover, the filesystem is actually a decent database for multiprocess communication. If every translation unit is compiled into an independent file, which is then combined into a single final output, there is no need to build any complex locking mechanisms or the like and you still get to take advantage of the embarrassingly parallel nature of compiling.
Incremental compilation is an incredibly important tool. If you make a small change to one file, it's frequently not necessary to rebuild most of the code. Making the output of individual file compilations work in a way that allows incremental compilation to happen requires basically building .o files--and there's very little savings to be had by not emitting them to disk.
Finally, I'll note that very frequently, debug builds of large applications cannot fit in RAM. Debugging symbols bloat builds tremendously, especially in intermediate object form (since many symbols end up needing to be duplicated in every single .o file). A debug build of a large application may take up 80GB in disk space, to build a binary that (without debug symbols) would be perhaps 100MB in size.
Keeping everything in RAM just isn't feasible at scale.
It seems odd to me that we still require .o files at all. Given the final program needs to be storable in RAM + Virtual Memory it surprises me that we still need the intermediate step of pushing to the file system only to then immediately reopen and merge those files. Does someone have more info on this? Or is the reason just legacy? Or is the reason just some ideological “single responsibility” thing?
Well, you might know that all those source files belong to exactly one executable (or library), but how would the compiler know?
Earlier quoted context omitted.
It's not, if it's contained to just the linker itself (which it is now), that's not why I stopped using it. I did so because it seems like they don't understand that much about AGPL and licensing in general and could change their license terms at any point to say something like "we claim AGPL propagates to the linker's output" which is very legally tenuous itself to claim so.
Tenuous or not, I believe GCC explicitly has a licensing exception that states that compiler output is not considered a derivative work of the compiler, and thus must also be licensed under the GPL. So the GNU/FSF folks at least thought it was a concerning enough legal idea to explicitly account for it. Not sure we can say that a linker is the same as a compiler in this sense, but if so, maybe it is indeed worrisome.
The standard compiler license exception (this applies to LLVM to, e.g.) says that any such code that gets combined in with your application code doesn't count. Note that it's still a potential license violation to use that code elsewhere (say, using those routines in another compiler).
This isn't a concern for linkers because linkers don't really provide anything in the way of code, everything being provided by the compiler as a compiler or language support library. The largest code it might add to your program is probably the PLT stub code, at best a couple of instructions long.
Previous discussions: https://news.ycombinator.com/item?id=26233244 https://news.ycombinator.com/item?id=29568454 https://news.ycombinator.com/item?id=33584651 https://news.ycombinator.com/item?id=34141912
Mold linker: targeting macOS/iOS now requires a commercial license - https://news.ycombinator.com/item?id=34141912 - Dec 2022 (74 comments)
Mold linker may switch to a source-available license - https://news.ycombinator.com/item?id=33584651 - Nov 2022 (206 comments)
Mold linker creator considers changing the license - https://news.ycombinator.com/item?id=33495528 - Nov 2022 (19 comments)
Mold/macOS is 11 times faster than the Apple's default linker to link Chrome - https://news.ycombinator.com/item?id=31769699 - June 2022 (116 comments)
Using the mold linker for fun and 3x-8x link time speedups - https://news.ycombinator.com/item?id=31604772 - June 2022 (41 comments)
Using the mold linker for fun and 3x-8x link time speedups - https://news.ycombinator.com/item?id=31592678 - June 2022 (1 comment)
Mold 1.0: the first stable and production-ready release of the high-speed linker - https://news.ycombinator.com/item?id=29568454 - Dec 2021 (65 comments)
Mold: A Modern Linker - https://news.ycombinator.com/item?id=26233244 - Feb 2021 (122 comments)
Mold: A Modern Linker - https://news.ycombinator.com/item?id=25410312 - Dec 2020 (1 comment)
It seems odd to me that we still require .o files at all. Given the final program needs to be storable in RAM + Virtual Memory it surprises me that we still need the intermediate step of pushing to the file system only to then immediately reopen and merge those files. Does someone have more info on this? Or is the reason just legacy? Or is the reason just some ideological “single responsibility” thing?
You don't. I recommend you learn about unity builds, and how major build systems support toggling them at the project and subproject level.
The main reason why most people haven't heard about the concept and those who did the majority doesn't bother with it is that a) you have little to nothing to gain by them, b) you throw incremental builds out if the window, c) you ruin internal linkage and thus can introduce hard to track errors.
Also, it makes no sense at all to argue how the released software needs to run in memory to justify aspects related to how the software is built. At most you have arguments over code bloat and premature optimization, but at what cost?
I always wanted to use mold in the build system I did last year: the metrics are fucking impressive. There were subtle differences to gold and ldd that I didn’t have time to chase down, but it seems like the future.
I can't say I've noticed much of a difference myself, maybe 20% faster at best.