Assuming that the linker script(s) used by a program don’t change very often, it might be interesting to compile them, in effect building a custom linker specialized to the script(s). Most programs don’t specify a linker script so maybe the linker can be specialized just to the usual scripts, and odd cases can use post-link binary editing.
Mold: A Modern Linker
71–80 of 125 posts
Re: Mold: A Modern Linker
#72I would use this in a heartbeat if you make execute-only a first-class feature. That means segments with E only, no reading.
My experience with every linker so far is that to have XO I will need to specify the memory layout in a linker script manually. It's not as nice as simple linker argument, if that is possible.
Re: Mold: A Modern Linker
#73From a marketing perspective, "mold" meaning "a form used to cast an object from liquid" is a lot more appealing than "green fungus growing on bread." A mold for casting objects is also a lot closer, metaphorically speaking, to what a linker does. I honestly thought that was the meaning the author was trying to evoke before I saw the picture on the github page.
For the record, a candidate for another name was "weld" as it joins pieces of data into a single binary. That's I think a good name, but I couldn't come up with a backronym.
wildly exhilarated link editor
Re: Mold: A Modern Linker
#74From a marketing perspective, "mold" meaning "a form used to cast an object from liquid" is a lot more appealing than "green fungus growing on bread." A mold for casting objects is also a lot closer, metaphorically speaking, to what a linker does. I honestly thought that was the meaning the author was trying to evoke before I saw the picture on the github page.
For the record, a candidate for another name was "weld" as it joins pieces of data into a single binary. That's I think a good name, but I couldn't come up with a backronym.
That's fine, because it's not exactly clear what "ld" stands for to begin with. If anyone asks, you could just say that the W and E stand for "Weally Efficient".
Re: Mold: A Modern Linker
#75If we mmap a lot of files, _exit(2) is not instantaneous but takes a few hundred milliseconds because the kernel has to clean up a lot of resources. As a workaround, we should organize the linker command as two processes; the first process forks the second process, and the second process does the actual work. As soon as the second process writes a result file to a filesystem, it notifies the first process, and the first process exits. The second process can take time to exit, because it is not an interactive process.
Is this safe? Are you sure that the _exit() delays are not part of the kernel committing all the pending mmap I/O to the buffer caches? If a build script links a binary using this, and then immediately executes it, the second (background process) might still not be finished. Is it guaranteed that all of the mmap I/O will be visible - or will the binary appear incomplete?
I don't know the answer to this myself - it would be clear cut if the linker was using write() calls, because UNIX guarantees that future read() calls will see the results, but the ordering guarantees with mmap I/O are far more loose, I believe.
Re: Mold: A Modern Linker
#76If we mmap a lot of files, _exit(2) is not instantaneous but takes a few hundred milliseconds because the kernel has to clean up a lot of resources. As a workaround, we should organize the linker command as two processes; the first process forks the second process, and the second process does the actual work. As soon as the second process writes a result file to a filesystem, it notifies the first process, and the fi…
It is safe because the child process calls munmap before telling its parent process to exit. munmap is guaranteed to act as a commit operation. Alternatively, you can call msync (https://man7.org/linux/man-pages/man2/msync.2.html) if you want to keep it mmapped.
Re: Mold: A Modern Linker
#77I would use this in a heartbeat if you make execute-only a first-class feature. That means segments with E only, no reading. My experience with every linker so far is that to have XO I will need to specify the memory layout in a linker script manually. It's not as nice as simple linker argument, if that is possible.
By execute-only segment, you mean a segment which is not readable but executable, right?
If so, that's a relatively new CPU security feature. I think some ARM processors support it, but AFAIK x86 doesn't support it at the moment. On x86, if you make a page executable, it automatically makes the page readable. R and X bits are not separated in the page table. I bet Intel and AMD will ad NR bit (no read bit - analogous to NX bit) soon, though.
Re: Mold: A Modern Linker
#78Overwriting executables during a build rather than copying them at the end can be problematic if a build artifact is used to bootstrap itself (like a compiler). If the build fails you can't try again.
Re: Mold: A Modern Linker
#79If we mmap a lot of files, _exit(2) is not instantaneous but takes a few hundred milliseconds because the kernel has to clean up a lot of resources. As a workaround, we should organize the linker command as two processes; the first process forks the second process, and the second process does the actual work. As soon as the second process writes a result file to a filesystem, it notifies the first process, and the fi…
It is safe because the child process calls munmap before telling its parent process to exit. munmap is guaranteed to act as a commit operation. Alternatively, you can call msync ( https://man7.org/linux/man-pages/man2/msync.2.html ) if you want to keep it mmapped.
Linux gives much stronger guarantees than POSIX here. I wonder if you save measurable time by skipping munmap.
Re: Mold: A Modern Linker
#80I’m surprised by the statement that ‘cat’ is slow because it’s not multithreaded. Isn’t ‘cat’ io-bound?
I think his point is that you can match `cat`s speed because it is IO bound and you can make a linker IO bound by using lots of threads.
It used to be lore in the industry that any decent linker is IO bound, including the single-threaded ones.