Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mold: A Modern Linker

github.com

Post reply on HN
Print view View original

111–120 of 125 posts

Re: Mold: A Modern Linker

#111

Post by bla3 »

ComputerGuru wrote: post #102
bla3 wrote: post #93

Earlier quoted context omitted.

What about posix_spawn() with POSIX_SPAWN_USEVFORK? That saves some of the overhead. See eg https://github.com/rtomayko/posix-spawn#benchmarks

posix_spawn is just a wrapper that takes care of setting common parameters for newly forked instances (eg pgrp) and prevents you from doing things that might be overly unsafe or could break vfork from being used in its optimized form. It’s implanted at the libc level, so it’s not a magic syscall that moves the burden of process spawning to the kernel.

Yes, but with that define it wraps vfork instead of fork.

Re: Mold: A Modern Linker

#112

Post by [deleted] »

ArchOversight wrote: post #109
malkia wrote: post #96

Earlier quoted context omitted.

Or this: for (std::string_view arg : config.version_script) parse_version_script(std::string(arg));

`parse_version_script` is defined as taking an `std::string`, std::string() over an `std::string_view` will create a new std::string with a copy of the data from arg.

[deleted]

Re: Mold: A Modern Linker

#113

Post by AndyKelley »

rui314 wrote: post #50
PoignardAzur wrote: post #45

I wonder how rui314's assertion that incremental linking is a poor tradeoff squares with Zig's decision[1] to build its own linker with "in-place binary patching". I assume part of the difference is that Zig has complete control over its environment, whereas Mold is trying to be a general-purpose linker, but still, I wonder if there's some insight to be gained from crosspollination there. (like, maybe Mold could have…

I got an impression that (so take it with a grain of salt), for incremental linking, Zig emits code that always uses PLT and GOT to access global functions and variables. Later, in order to replace functions or variables, Zig rewrites only PLT or GOT entries. On the other hand, by default, gcc or clang emits code that does not use GOT or PLT, which makes the situation much more complicated. In addition to that, maybe…

Your impression is correct.

When the Zig compiler is asked to produce an executable, using only .zig source code, it is in full control of the entire compilation process, frontend to backend, so it can make decisions that ease the requirements of its linker, in order to facilitate incremental compilation and linking. For example, when linking pure Zig projects, there are no relocations; the code is generated directly in place in the final executable, almost as if there is no linker step. However, when asked to link against third party objects or static libraries, Zig must compromise some of these advantages. Currently, in this situation, Zig will fall back to doing the incremental compilation of .zig code into an object file, and then invoke LLD (via the zig executable invoking itself as a child process) to link the objects together. As the Zig self-hosted linker gains functionality, this fallback will happen less often; instead the compromise will be in the code paths taken in the linker, based on what assumptions it can make about the linking requirements that are required for a given job. The long term plan is to eliminate the dependency on LLD altogether.

Side note - mold is a brilliant project! Thank you for making it and pushing the state of the art forward! Also I love the logo.

Re: Mold: A Modern Linker

#114

Post by siithy »

rui314 wrote: post #56
jdblair wrote: post #13

From a marketing perspective, "mold" meaning "a form used to cast an object from liquid" is a lot more appealing than "green fungus growing on bread." A mold for casting objects is also a lot closer, metaphorically speaking, to what a linker does. I honestly thought that was the meaning the author was trying to evoke before I saw the picture on the github page.

For the record, a candidate for another name was "weld" as it joins pieces of data into a single binary. That's I think a good name, but I couldn't come up with a backronym.

How about "wildly experimental linking device"?

Edit: Had fun thinking up a couple others:

"whimsically eclectic logical design"

"willfully egregious lackadaisical decision"

"wise element location director"

"world exploring layout detector"

"wrong-headed exasperating liability defender"

"workaday execution layer developer"

"wonderfully elegant logistical delegator"

Re: Mold: A Modern Linker

#115

Post by brigandish »

kleiba wrote: post #30
felixr wrote: post #22

Earlier quoted context omitted.

I was going to say: but the casting template is spelled 'mould' not 'mold'. Looked it up and realised that the US spelling is actually also 'mold' https://www.oxfordlearnersdictionaries.com/definition/englis...

Moreover, the fungus is spelled 'mould' in British English: https://dictionary.cambridge.org/dictionary/english/mould

Why wouldn't the fungus be spelt mould in British English? I'm just wondering if there's a reason beyond the natural surprise of finding variations, not trying to be argumentative.

I've just been interrupred while writing this to be told that America and Britain number calendar weeks differently (Britain follows ISO[1]) and that Apple's calendar is fixed to the US version. It never ends…!

[1] https://en.wikipedia.org/wiki/ISO_week_date

Re: Mold: A Modern Linker

#116

Post by rui314 »

amluto wrote: post #91
rui314 wrote: post #83

Earlier quoted context omitted.

Is that documented?

I would have said yes, but I can’t find it. That being said, Linux has a “unified page cache”, and MAP_SHARED is coherent with read(2) and write(2), at least on any local filesystem (not sure about FUSE) and when direct IO is not involved. That being said, I could easily believe that largeish pwrite(2) calls would be comparably fast compared to mmap, since mmap needs to play with page tables, and page faults on x86 a…

I once counted the number of 4 KiB blocks that has at least one relocation. I used Chrome as a sample. It turned out that almost all 4 KiB blocks have at least one relocation. They mutate everywhere.

Re: Mold: A Modern Linker

#117

Post by rui314 »

fwsgonzo wrote: post #105
rui314 wrote: post #77

Earlier quoted context omitted.

By execute-only segment, you mean a segment which is not readable but executable, right? If so, that's a relatively new CPU security feature. I think some ARM processors support it, but AFAIK x86 doesn't support it at the moment. On x86, if you make a page executable, it automatically makes the page readable. R and X bits are not separated in the page table. I bet Intel and AMD will ad NR bit (no read bit - analogous…

Intel slide deck here: https://linuxplumbersconf.org/event/4/contributions/283/atta... If they really support it, I didn't pay too close attention, but at least it's on their mind.

Ooh, I didn't know that x86 had such feature. But is this virtualization only?

Re: Mold: A Modern Linker

#118

Post by rui314 »

malkia wrote: post #95
malkia wrote: post #94

Since perf is at utmost importance for this project, and intern has been found to be used a lot, maybe a pinch of small optimization is to move the static ConcurrentMap out of the function, hence avoid atomic check on whether it's initialized - static Symbol *intern(std::string_view name) { static ConcurrentMap map; return map.insert(name, {name}); } to static ConcurrentMap map; static Symbol *intern(std::string_view…

Wait this code can't work - as you holding only a string_view in the Symbol... // __start_ and __stop_ symbols for (OutputChunk *chunk : chunks) { if (is_c_identifier(chunk->name)) { start(Symbol::intern("__start_" + std::string(chunk->name)), chunk); stop(Symbol::intern("__stop_" + std::string(chunk->name)), chunk); } }

Ah, that's a bug. Thank you for finding it!

Re: Mold: A Modern Linker

#119

Post by ncmncm »

Standard C++ has parallel algorithm primitives so there no need to depend on TBB anymore.

You can give each thread its own malloc that just mmaps what it needs, if you are leaking everything anyway.

This is a case where using a raw new() and raw pointers is much better than using a smart pointer, because touching things to run destructors unnecessarily is itself expensive.

Re: Mold: A Modern Linker

#120

Post by ncmncm »

amluto wrote: post #79
rui314 wrote: post #76

Earlier quoted context omitted.

It is safe because the child process calls munmap before telling its parent process to exit. munmap is guaranteed to act as a commit operation. Alternatively, you can call msync ( https://man7.org/linux/man-pages/man2/msync.2.html ) if you want to keep it mmapped.

Linux gives much stronger guarantees than POSIX here. I wonder if you save measurable time by skipping munmap.

munmap is often a remarkably slow operation, if your process is multi-threaded, because of TLB shootdowns; on each munmap, all the other threads get paused and their page map caches get trashed, each time.

It is usually much better to have multiple regular processes, instead of threads, that only share chosen mappings, if you want to use munmap. Or, you can terminate and join all your threads before you start munmapping.

Post reply on HN