Live data from Hacker News

Mold: A Modern Linker

github.com

41–50 of 125 posts

Re: Mold: A Modern Linker

#41

Damn, the idea of string interning symbol names in a preload pass is tight. That's one of those sentences that you read and from then on it's just obviously the right way to write a linker.

I was surprised to learn that this isn't how linkers work. Apparently one of the reasons c++ template code is so slow to link is long symbol names.

Re: Mold: A Modern Linker

#42
I am not sure that multi-threaded is the way to optimize file writes; I would have thought that leaving as much as possible up to the kernel, by using writev(2), splice(2), etc. would be best.

Re: Mold: A Modern Linker

#43
post #3

> I won't avoid Unix-ism when writing code (e.g. I'll probably use fork(2)). It’s probably fine to not avoid most Unix APIs but fork() is truly an exception here. Fork() is not friendly to any third party library you may use because their state may become invalidated after a fork but the library has no way to know if the process has been forked. This is especially bad if the library uses multi threading. The best way…

Author here. Good point. I ended up not using fork() without exec(), so that should be fine now, but here is my original plan to use fork(): I wanted to keep a linker process running as a daemon so that it doesn't read the same files over and over again. After loading input files, the linker becomes a daemon and calls fork() to create a worker process. Then the worker process does the rest of linking. In other word,…

I wonder if the tax could be reduced with huge pages. Much of the cost of COW is the large number of page faults, but with huge pages you could reduce the number of faults to 1.

Re: Mold: A Modern Linker

#44
So happy to see progress in linkers!

Can this one statically link shared libraries? That would be a great feature.

EDIT: I realize that such a feature would necessarily involve a fair amount of black magic. But it does not seem an impossible endeavor.

Re: Mold: A Modern Linker

#45
I wonder how rui314's assertion that incremental linking is a poor tradeoff squares with Zig's decision[1] to build its own linker with "in-place binary patching".

I assume part of the difference is that Zig has complete control over its environment, whereas Mold is trying to be a general-purpose linker, but still, I wonder if there's some insight to be gained from crosspollination there.

(like, maybe Mold could have an "--incremental" option with associate documentation explaining that this option is only beneficial if the input object files follow some specific guidelines)

[1] https://kristoff.it/blog/zig-new-relationship-llvm/

Re: Mold: A Modern Linker

#46

...

There might be a 'best of both worlds' idea in there: If mold mmaps the input files instead of reading them, the linker would not trash all cached files, and be faster itself as it reuses already cached files. Now the author seems a smart fellow, so maybe he did just that already, I didn't check the source.

mold uses only mmap for file IO. Not only input files but also an output file are mapped to memory using mmap(2).

Re: Mold: A Modern Linker

#47
post #32
post #19

Earlier quoted context omitted.

I agree with you on both points but when building large complex applications (or tools!) this stuff tends to unintentionally creep up over time and it’s not fun to debug non-deterministic bugs due to an opaquely broken implicit contract between you and a poorly written library that got pulled in transitively. Sometimes those libraries are unavoidable closed source system libraries ( https://bugs.python.org/issue33725…

It's a moot point since the author already said he's not using fork(), but C and C++ apps don't transitively pull in dependencies, especially in the open source world. That kind of thinking comes from node.js and Rust as far as I can tell. There is never ever a situation where a program like GCC or Clang will acquire a third party dependency without a commit that explicitly changes the build system. The Python exampl…

Of course C and C++ apps transitively pull in dependencies.

First, pkg-config exists and most projects on Unixy OSes use it these days. It doesn't really make much sense to argue "it all has to be specified on the command-line" when the command-line the developer cares about looks like this:

gcc `pkg-config --cflags gtk+-3.0` -o test test.c `pkg-config --libs gtk+-3.0`

(That adds things like `-lcairo` which aren't at all visible at first glance)

Second, why do you think things like libwayland-client, libfreetype, and libexpat show up in the ldd output? They're certainly not in the pkg-config output.

Likewise, nothing says a library has to bump its major version number when it adds a new backend dependency, which means that a distro may add a transitive dependency through a shared library update that doesn't correspond to you bumping your version.

(You can gain new backend dependencies without even recompiling anything you're responsible for.)

Heck, this post explicitly touches on that disconnect between what you add to your build plan and what ldd lists for C and C++ codebases.

https://wiki.alopex.li/LetsBeRealAboutDependencies

Re: Mold: A Modern Linker

#48
post #36

Earlier quoted context omitted.

On the other hand, Rust was apparently named after the fungus, not iron oxide. There seems to be a new category of low-level tools named after life forms that grow in ecological niches :D

How interesting. Doesn't the fungus get its name from the oxidized metal though and not the other way around? Mold the form and mold the fungus seem etymologically unrelated, however.

> Doesn't the fungus get its name from the oxidized metal though and not the other way around?

Huh, apparently so! So Rust-the-language isn't necessarily named after oxidized metal, but it _is_ named after something that's named after oxidized metal.

Re: Mold: A Modern Linker

#49
post #30
post #22

Earlier quoted context omitted.

I was going to say: but the casting template is spelled 'mould' not 'mold'. Looked it up and realised that the US spelling is actually also 'mold' https://www.oxfordlearnersdictionaries.com/definition/englis...

Moreover, the fungus is spelled 'mould' in British English: https://dictionary.cambridge.org/dictionary/english/mould

You gotta love the English language. Contradictions abound.

Re: Mold: A Modern Linker

#50

I wonder how rui314's assertion that incremental linking is a poor tradeoff squares with Zig's decision[1] to build its own linker with "in-place binary patching". I assume part of the difference is that Zig has complete control over its environment, whereas Mold is trying to be a general-purpose linker, but still, I wonder if there's some insight to be gained from crosspollination there. (like, maybe Mold could have…

I got an impression that (so take it with a grain of salt), for incremental linking, Zig emits code that always uses PLT and GOT to access global functions and variables. Later, in order to replace functions or variables, Zig rewrites only PLT or GOT entries.

On the other hand, by default, gcc or clang emits code that does not use GOT or PLT, which makes the situation much more complicated.

In addition to that, maybe you don't have to support all ELF fancy features if you know that you are linking Zig programs? I'm not familiar with Zig, but I can say that some kind of minor feature, such as weak symbol, can make incremental linking a lot harder.

Post reply on HN