Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mold: A Modern Linker

github.com

Post reply on HN
Print view View original

101–110 of 125 posts

Re: Mold: A Modern Linker

#101

Post by chubot »

ssokolow wrote: post #47
chubot wrote: post #32

Earlier quoted context omitted.

It's a moot point since the author already said he's not using fork(), but C and C++ apps don't transitively pull in dependencies, especially in the open source world. That kind of thinking comes from node.js and Rust as far as I can tell. There is never ever a situation where a program like GCC or Clang will acquire a third party dependency without a commit that explicitly changes the build system. The Python exampl…

Of course C and C++ apps transitively pull in dependencies. First, pkg-config exists and most projects on Unixy OSes use it these days. It doesn't really make much sense to argue "it all has to be specified on the command-line" when the command-line the developer cares about looks like this: gcc `pkg-config --cflags gtk+-3.0` -o test test.c `pkg-config --libs gtk+-3.0` (That adds things like `-lcairo` which aren't at…

That is a good point... I overstated the case, although I would say the GCC and Clang examples in particular are true. I guess the stronger argument is that compilers are more of a special case low in the dependency stack, as opposed to apps (and especially GUI apps). Also, I really hope none of those dependencies start long-lived threads without the app's knowledge, though there's nothing stopping them from doing so.

Re: Mold: A Modern Linker

#102

Post by ComputerGuru »

bla3 wrote: post #93
rui314 wrote: post #3

Earlier quoted context omitted.

Author here. Good point. I ended up not using fork() without exec(), so that should be fine now, but here is my original plan to use fork(): I wanted to keep a linker process running as a daemon so that it doesn't read the same files over and over again. After loading input files, the linker becomes a daemon and calls fork() to create a worker process. Then the worker process does the rest of linking. In other word,…

What about posix_spawn() with POSIX_SPAWN_USEVFORK? That saves some of the overhead. See eg https://github.com/rtomayko/posix-spawn#benchmarks

posix_spawn is just a wrapper that takes care of setting common parameters for newly forked instances (eg pgrp) and prevents you from doing things that might be overly unsafe or could break vfork from being used in its optimized form. It’s implanted at the libc level, so it’s not a magic syscall that moves the burden of process spawning to the kernel.

Re: Mold: A Modern Linker

#103

Post by zamalek »

k_bx wrote: post #26
brokencode wrote: post #8

Is it possible to use alternate linkers with Rust? Anything that helps Rust compile faster is pretty cool, especially if it allows better multithreaded scaling.

Yes, put this in your ~/.cargo/config.toml: [target.x86_64-unknown-linux] linker = "ld.lld"

And it does help compilation times quite a bit.

Re: Mold: A Modern Linker

#104

Post by zamalek »

thethirdone wrote: post #31
kasabali wrote: post #29

...

Not caring amount memory leaks is not the same as not caring about memory consumption. The next line shows exactly why the choice is made. > It is because most objects that are allocated during an execution of mold are needed until the very end of the program. Using free would not affect that memory consumption much because most consumption would be freed at the end of the program.

Raymond Chen has an amusing story about this[1]:

> Since the missile will explode when it hits its target or at the end of its flight, the ultimate in garbage collection is performed without programmer intervention.

[1]: https://devblogs.microsoft.com/oldnewthing/20180228-00/?p=98...

Re: Mold: A Modern Linker

#105

Post by fwsgonzo »

rui314 wrote: post #77
fwsgonzo wrote: post #72

I would use this in a heartbeat if you make execute-only a first-class feature. That means segments with E only, no reading. My experience with every linker so far is that to have XO I will need to specify the memory layout in a linker script manually. It's not as nice as simple linker argument, if that is possible.

By execute-only segment, you mean a segment which is not readable but executable, right? If so, that's a relatively new CPU security feature. I think some ARM processors support it, but AFAIK x86 doesn't support it at the moment. On x86, if you make a page executable, it automatically makes the page readable. R and X bits are not separated in the page table. I bet Intel and AMD will ad NR bit (no read bit - analogous…

Intel slide deck here: https://linuxplumbersconf.org/event/4/contributions/283/atta...

If they really support it, I didn't pay too close attention, but at least it's on their mind.

Re: Mold: A Modern Linker

#106

Post by CyberRabbi »

chubot wrote: post #32
_dh54 wrote: post #19

Earlier quoted context omitted.

I agree with you on both points but when building large complex applications (or tools!) this stuff tends to unintentionally creep up over time and it’s not fun to debug non-deterministic bugs due to an opaquely broken implicit contract between you and a poorly written library that got pulled in transitively. Sometimes those libraries are unavoidable closed source system libraries ( https://bugs.python.org/issue33725…

It's a moot point since the author already said he's not using fork(), but C and C++ apps don't transitively pull in dependencies, especially in the open source world. That kind of thinking comes from node.js and Rust as far as I can tell. There is never ever a situation where a program like GCC or Clang will acquire a third party dependency without a commit that explicitly changes the build system. The Python exampl…

In my example, using Apple’s system carbon libraries at the C level transitively pulls in their multithreading framework. It does happen. Everything that can break will break, that attitude is necessary to build robust software.

Re: Mold: A Modern Linker

#107

Post by CyberRabbi »

ComputerGuru wrote: post #102
bla3 wrote: post #93

Earlier quoted context omitted.

What about posix_spawn() with POSIX_SPAWN_USEVFORK? That saves some of the overhead. See eg https://github.com/rtomayko/posix-spawn#benchmarks

posix_spawn is just a wrapper that takes care of setting common parameters for newly forked instances (eg pgrp) and prevents you from doing things that might be overly unsafe or could break vfork from being used in its optimized form. It’s implanted at the libc level, so it’s not a magic syscall that moves the burden of process spawning to the kernel.

This doesn’t change your core point but just to be thorough, it’s actually a system call on a few systems, notably macOS.

Re: Mold: A Modern Linker

#108

Post by steveklabnik »

agumonkey wrote: post #85
Twisol wrote: post #48

Earlier quoted context omitted.

> Doesn't the fungus get its name from the oxidized metal though and not the other way around? Huh, apparently so! So Rust-the-language isn't necessarily named after oxidized metal, but it _is_ named after something that's named after oxidized metal.

And what's the relationship between the programming language and fungi ? is it a root growing invading the world concept ? is it a pun on 'fun'-'gi' ?

https://www.reddit.com/r/rust/comments/27jvdt/internet_archa... is the primary source here. Quotes:

> fungi are amazingly robust

> to start, they are distributed organisms. not single cellular, but also no single point of failure.

However, I will note that graydon uses words like "I think I" and "I remember kinda" and everyone says "yes this means this is 100% the source of the name" whereas I take it to be like, "this is one of many reasons that Rust is named Rust."

You also have to remember that Rust was a very different language in many senses back when Graydon was choosing the name, so allusions may not make sense now but may have then. Early Rust was much more erlang-like, which may make the above feel more relevant.

Re: Mold: A Modern Linker

#109

Post by ArchOversight »

malkia wrote: post #96
malkia wrote: post #95

Earlier quoted context omitted.

Wait this code can't work - as you holding only a string_view in the Symbol... // __start_ and __stop_ symbols for (OutputChunk *chunk : chunks) { if (is_c_identifier(chunk->name)) { start(Symbol::intern("__start_" + std::string(chunk->name)), chunk); stop(Symbol::intern("__stop_" + std::string(chunk->name)), chunk); } }

Or this: for (std::string_view arg : config.version_script) parse_version_script(std::string(arg));

`parse_version_script` is defined as taking an `std::string`, std::string() over an `std::string_view` will create a new std::string with a copy of the data from arg.

Re: Mold: A Modern Linker

#110

Post by ArchOversight »

malkia wrote: post #95
malkia wrote: post #94

Since perf is at utmost importance for this project, and intern has been found to be used a lot, maybe a pinch of small optimization is to move the static ConcurrentMap out of the function, hence avoid atomic check on whether it's initialized - static Symbol *intern(std::string_view name) { static ConcurrentMap map; return map.insert(name, {name}); } to static ConcurrentMap map; static Symbol *intern(std::string_view…

Wait this code can't work - as you holding only a string_view in the Symbol... // __start_ and __stop_ symbols for (OutputChunk *chunk : chunks) { if (is_c_identifier(chunk->name)) { start(Symbol::intern("__start_" + std::string(chunk->name)), chunk); stop(Symbol::intern("__stop_" + std::string(chunk->name)), chunk); } }

This does seem broken, since the temporary string that is created by the concatenation is short lived, and the std::string_view inside the Symbol won't hold on to it, just the pointer to the data.
Post reply on HN