Mold: A Modern Linker
github.com
Mold: A Modern Linker
1–10 of 125 posts
Re: Mold: A Modern Linker
#2> I won't avoid Unix-ism when writing code (e.g. I'll probably use fork(2)).
It’s probably fine to not avoid most Unix APIs but fork() is truly an exception here. Fork() is not friendly to any third party library you may use because their state may become invalidated after a fork but the library has no way to know if the process has been forked. This is especially bad if the library uses multi threading. The best way to avoid a random unintended bug later on is to only use fork() when you plan to execve() right after.
Re: Mold: A Modern Linker
#3> I won't avoid Unix-ism when writing code (e.g. I'll probably use fork(2)). It’s probably fine to not avoid most Unix APIs but fork() is truly an exception here. Fork() is not friendly to any third party library you may use because their state may become invalidated after a fork but the library has no way to know if the process has been forked. This is especially bad if the library uses multi threading. The best way…
Author here. Good point. I ended up not using fork() without exec(), so that should be fine now, but here is my original plan to use fork():
I wanted to keep a linker process running as a daemon so that it doesn't read the same files over and over again. After loading input files, the linker becomes a daemon and calls fork() to create a worker process. Then the worker process does the rest of linking. In other word, a daemon is a "clean" copy of a linker process image, and each child is specialized for each actual linker invocation.
It turned out that the linker runs much slower with fork() because of the overhead of copy-on-write. You cannot keep a fresh copy of a process just by calling fork() for free. There's a tax associated with it. I tried to workaround, but in the end I had to give up with the fork()-based worker process design.
Re: Mold: A Modern Linker
#4Damn, the idea of string interning symbol names in a preload pass is tight. That's one of those sentences that you read and from then on it's just obviously the right way to write a linker.
Re: Mold: A Modern Linker
#5> As an implementation strategy, we do not care about memory leak because we really can't save that much memory by doing precise memory management. It is because most objects that are allocated during an execution of mold are needed until the very end of the program. I'm sure this is an odd memory management scheme (or the lack thereof), but this is what LLVM lld does too.
The fact that someone does it wrong doesn't mean that we should do it wrong as well ;(
I don't think I like this approach. It may work now, but will probably seriously limit the possibilities in the future.
Re: Mold: A Modern Linker
#6> As an implementation strategy, we do not care about memory leak because we really can't save that much memory by doing precise memory management. It is because most objects that are allocated during an execution of mold are needed until the very end of the program. I'm sure this is an odd memory management scheme (or the lack thereof), but this is what LLVM lld does too. The fact that someone does it wrong doesn't…
If you think this approach is wrong, could you articulate the reasons why you think it is wrong? This is a classic memory management strategy... if a program is running as a batch program, all memory will be freed when the program exits. Any alternative memory management strategy would have to free and then reuse memory in order to show improvement. If it's a small amount of memory freed, or if the memory is unlikely to be reused, the benefits of freeing memory are smaller and it may actually slow the program down.
The fact that this program can successfully link Chrome means that we have fairly solid baseline performance metrics we can use for "big" programs. Chrome is just about the largest program you might ever need to link.
Re: Mold: A Modern Linker
#7This page is extremely correct about how extremely under-documented linker scripts are. I wish I could learn more about them since you can do a lot of C/C++ magic.
Re: Mold: A Modern Linker
#8Is it possible to use alternate linkers with Rust? Anything that helps Rust compile faster is pretty cool, especially if it allows better multithreaded scaling.
Re: Mold: A Modern Linker
#9Is it possible to use alternate linkers with Rust? Anything that helps Rust compile faster is pretty cool, especially if it allows better multithreaded scaling.
I believe the Rust compiler can use an alternative linker, but I'm pretty sure that mold can't link Rust programs because it lacks lots of features. It's so experimental that I didn't even think about trying to link a Rust program with it.
Re: Mold: A Modern Linker
#10> I won't avoid Unix-ism when writing code (e.g. I'll probably use fork(2)). It’s probably fine to not avoid most Unix APIs but fork() is truly an exception here. Fork() is not friendly to any third party library you may use because their state may become invalidated after a fork but the library has no way to know if the process has been forked. This is especially bad if the library uses multi threading. The best way…
I can't think of any reason a linker needs to use many libraries, especially not one that start threads. A linker reads files and writes a file. It's a single-purpose tool, not a monolithic app with many dependencies.
Either way, threads should be controlled by the application and not libraries. Well-written libraries like sqlite and Lua are parameterized by I/O and concurrency. They don't read files and start threads behind your back.