Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mold: A Modern Linker

github.com

Post reply on HN
Print view View original

91–100 of 125 posts

Re: Mold: A Modern Linker

#91

Post by amluto »

rui314 wrote: post #83
amluto wrote: post #79

Earlier quoted context omitted.

Linux gives much stronger guarantees than POSIX here. I wonder if you save measurable time by skipping munmap.

Is that documented?

I would have said yes, but I can’t find it. That being said, Linux has a “unified page cache”, and MAP_SHARED is coherent with read(2) and write(2), at least on any local filesystem (not sure about FUSE) and when direct IO is not involved.

That being said, I could easily believe that largeish pwrite(2) calls would be comparably fast compared to mmap, since mmap needs to play with page tables, and page faults on x86 are expensive. MAP_POPULATE would also be worth trying if you’re not already using it.

I assume that copy_file_range(2) is out of the question due to relocations.

Re: Mold: A Modern Linker

#93

Post by bla3 »

rui314 wrote: post #3
CyberRabbi wrote: post #2

> I won't avoid Unix-ism when writing code (e.g. I'll probably use fork(2)). It’s probably fine to not avoid most Unix APIs but fork() is truly an exception here. Fork() is not friendly to any third party library you may use because their state may become invalidated after a fork but the library has no way to know if the process has been forked. This is especially bad if the library uses multi threading. The best way…

Author here. Good point. I ended up not using fork() without exec(), so that should be fine now, but here is my original plan to use fork(): I wanted to keep a linker process running as a daemon so that it doesn't read the same files over and over again. After loading input files, the linker becomes a daemon and calls fork() to create a worker process. Then the worker process does the rest of linking. In other word,…

What about posix_spawn() with POSIX_SPAWN_USEVFORK? That saves some of the overhead. See eg https://github.com/rtomayko/posix-spawn#benchmarks

Re: Mold: A Modern Linker

#94

Post by malkia »

Since perf is at utmost importance for this project, and intern has been found to be used a lot, maybe a pinch of small optimization is to move the static ConcurrentMap out of the function, hence avoid atomic check on whether it's initialized -

  static Symbol *intern(std::string_view name) {
    static ConcurrentMap map;
    return map.insert(name, {name});
  }
to

  static ConcurrentMap map;

  static Symbol *intern(std::string_view name) {
    return map.insert(name, {name});
  }
probably though it won't bring much, but for the sake of squeezing every bit out there (and it was the easiest I could find - lol)

Re: Mold: A Modern Linker

#95

Post by malkia »

malkia wrote: post #94

Since perf is at utmost importance for this project, and intern has been found to be used a lot, maybe a pinch of small optimization is to move the static ConcurrentMap out of the function, hence avoid atomic check on whether it's initialized - static Symbol *intern(std::string_view name) { static ConcurrentMap map; return map.insert(name, {name}); } to static ConcurrentMap map; static Symbol *intern(std::string_view…

Wait this code can't work - as you holding only a string_view in the Symbol...

  // __start_ and __stop_ symbols
  for (OutputChunk *chunk : chunks) {
    if (is_c_identifier(chunk->name)) {
      start(Symbol::intern("__start_" + std::string(chunk->name)), chunk);
      stop(Symbol::intern("__stop_" + std::string(chunk->name)), chunk);
    }
  }

Re: Mold: A Modern Linker

#96

Post by malkia »

malkia wrote: post #95
malkia wrote: post #94

Since perf is at utmost importance for this project, and intern has been found to be used a lot, maybe a pinch of small optimization is to move the static ConcurrentMap out of the function, hence avoid atomic check on whether it's initialized - static Symbol *intern(std::string_view name) { static ConcurrentMap map; return map.insert(name, {name}); } to static ConcurrentMap map; static Symbol *intern(std::string_view…

Wait this code can't work - as you holding only a string_view in the Symbol... // __start_ and __stop_ symbols for (OutputChunk *chunk : chunks) { if (is_c_identifier(chunk->name)) { start(Symbol::intern("__start_" + std::string(chunk->name)), chunk); stop(Symbol::intern("__stop_" + std::string(chunk->name)), chunk); } }

Or this:

for (std::string_view arg : config.version_script) parse_version_script(std::string(arg));

Re: Mold: A Modern Linker

#97

Post by malkia »

malkia wrote: post #96
malkia wrote: post #95

Earlier quoted context omitted.

Wait this code can't work - as you holding only a string_view in the Symbol... // __start_ and __stop_ symbols for (OutputChunk *chunk : chunks) { if (is_c_identifier(chunk->name)) { start(Symbol::intern("__start_" + std::string(chunk->name)), chunk); stop(Symbol::intern("__stop_" + std::string(chunk->name)), chunk); } }

Or this: for (std::string_view arg : config.version_script) parse_version_script(std::string(arg));

Although if somehow nothing gets really freed, and std::string() is kept intact after calling the destructor (e.g. it's data() is still valid) then it'll work :)

Re: Mold: A Modern Linker

#98

Post by spacechild1 »

enriquto wrote: post #59
spacechild1 wrote: post #57

Earlier quoted context omitted.

> Equivalently, a tool for converting .a to .so Well, an .a file is just an archive of object files. Turning several object files into a single binary is pretty much what a linker does. However, you can't really automate this because object files might have external dependencies and the linker needs to know what these dependencies are.

> you can't really automate this because object files might have external dependencies and the linker needs to know what these dependencies are If it can be done at runtime of the program, I guess it can be done at runtime of the linker, doesn't it?

Technically, Linux and macOS allow to build a shared library with undefined symbols and let the loader figure it out, but I wouldn't recommend it (it's easy to miss linker errors). On Windows, however, this is not possible.

EDIT: also, building with unresolved symbols requires the host application to know about and link all the required external libraries, which is usually not what you want...

Re: Mold: A Modern Linker

#99

Post by spacechild1 »

higerordermap wrote: post #87
enriquto wrote: post #59

Earlier quoted context omitted.

> you can't really automate this because object files might have external dependencies and the linker needs to know what these dependencies are If it can be done at runtime of the program, I guess it can be done at runtime of the linker, doesn't it?

Yeah I have the same question. lld can list the dependencies right?

Yes, because someone put them there ;-)

Re: Mold: A Modern Linker

#100

Post by GrinningFool »

hobofan wrote: post #21
rui314 wrote: post #18

Earlier quoted context omitted.

Author here. Haha, that's perhaps true. But at the same time, it seems like a tradition to give a silly name (e.g. "git") to a tool, and I actually like that name and the image to show that I'm not too serious. This is a fun project but not ready for production use.

> that name and the image to show that I'm not too serious. This is a fun project but not ready for production use. I'm not sure if that image is the best way to communicate that status, given that the sudo sandwich logo exists (which coincidentally bears some resemblance to your moldy bread). A big bold "not ready for production" at the top of the README is probably a better way to achieve that.

The author has no such obligation, and "ready for production" is something that you would want to verify for yourself based on your evaluation of the project and your requirements.
Post reply on HN