Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mold: A Modern Linker

github.com

Post reply on HN
Print view View original

51–60 of 125 posts

Re: Mold: A Modern Linker

#51

Post by eek04_ »

dataflow wrote: post #12
rui314 wrote: post #3

Earlier quoted context omitted.

Author here. Good point. I ended up not using fork() without exec(), so that should be fine now, but here is my original plan to use fork(): I wanted to keep a linker process running as a daemon so that it doesn't read the same files over and over again. After loading input files, the linker becomes a daemon and calls fork() to create a worker process. Then the worker process does the rest of linking. In other word,…

> in the end I had to give up with the fork()-based worker process design. Honestly that's a good thing, or your program just fundamentally couldn't possibly ever work on native Windows!

Not possibly working on Windows is, for some of us, a feature. I have no wish to support that ecosystem due to the past ethical violations that founded it, and making it impossible to do so would avoid having to deal with it.

Re: Mold: A Modern Linker

#52

Post by rui314 »

teddyh wrote: post #42

I am not sure that multi-threaded is the way to optimize file writes; I would have thought that leaving as much as possible up to the kernel, by using writev(2), splice(2), etc. would be best.

I chose mmap based on benchmarking. Writing a 2 GiB memory buffer to a file using write(2) was slower than directly constructing file contents to mmap'ed memory region.

What is more interesting is the real bottleneck was not about mmap vs write(v) but in the filesystem. If you create a fresh file and write 2 GiB of data to that file, it takes like 700 milliseconds on my machine (ext4 fs), but if you write the same amount of data to an existing 2 GiB file, the IO speed doubles. So, the filesystem's performance to allocate new disk blocks seems to limit the performance of my linker. That reminded me of the axiom: don't guess about performance but measure.

Re: Mold: A Modern Linker

#53

Post by rui314 »

enriquto wrote: post #44

So happy to see progress in linkers! Can this one statically link shared libraries? That would be a great feature. EDIT: I realize that such a feature would necessarily involve a fair amount of black magic. But it does not seem an impossible endeavor.

Are you asking if mold can link an executable file and its depending .so files into a single binary? If so, neither mold nor other major linkers can't do that.

Re: Mold: A Modern Linker

#54

Post by enriquto »

rui314 wrote: post #53
enriquto wrote: post #44

So happy to see progress in linkers! Can this one statically link shared libraries? That would be a great feature. EDIT: I realize that such a feature would necessarily involve a fair amount of black magic. But it does not seem an impossible endeavor.

Are you asking if mold can link an executable file and its depending .so files into a single binary? If so, neither mold nor other major linkers can't do that.

Exactly. That would be a pretty big deal and definitely a killer feature. Equivalently, a tool for converting .a to .so

Re: Mold: A Modern Linker

#55

Post by koolba »

andrewshadura wrote: post #35
hobofan wrote: post #21

Earlier quoted context omitted.

> that name and the image to show that I'm not too serious. This is a fun project but not ready for production use. I'm not sure if that image is the best way to communicate that status, given that the sudo sandwich logo exists (which coincidentally bears some resemblance to your moldy bread). A big bold "not ready for production" at the top of the README is probably a better way to achieve that.

Or it could be "bold".

Or “weld”.

Which I think might just be the greatest name for a new linker.

Re: Mold: A Modern Linker

#56

Post by rui314 »

jdblair wrote: post #13

From a marketing perspective, "mold" meaning "a form used to cast an object from liquid" is a lot more appealing than "green fungus growing on bread." A mold for casting objects is also a lot closer, metaphorically speaking, to what a linker does. I honestly thought that was the meaning the author was trying to evoke before I saw the picture on the github page.

For the record, a candidate for another name was "weld" as it joins pieces of data into a single binary. That's I think a good name, but I couldn't come up with a backronym.

Re: Mold: A Modern Linker

#57

Post by spacechild1 »

enriquto wrote: post #54
rui314 wrote: post #53

Earlier quoted context omitted.

Are you asking if mold can link an executable file and its depending .so files into a single binary? If so, neither mold nor other major linkers can't do that.

Exactly. That would be a pretty big deal and definitely a killer feature. Equivalently, a tool for converting .a to .so

> Equivalently, a tool for converting .a to .so

Well, an .a file is just an archive of object files. Turning several object files into a single binary is pretty much what a linker does. However, you can't really automate this because object files might have external dependencies and the linker needs to know what these dependencies are.

Re: Mold: A Modern Linker

#58

Post by pjmlp »

teddyh wrote: post #42

I am not sure that multi-threaded is the way to optimize file writes; I would have thought that leaving as much as possible up to the kernel, by using writev(2), splice(2), etc. would be best.

That is only a solution if you want to hardcore it to a specific OS.

Re: Mold: A Modern Linker

#59

Post by enriquto »

spacechild1 wrote: post #57
enriquto wrote: post #54

Earlier quoted context omitted.

Exactly. That would be a pretty big deal and definitely a killer feature. Equivalently, a tool for converting .a to .so

> Equivalently, a tool for converting .a to .so Well, an .a file is just an archive of object files. Turning several object files into a single binary is pretty much what a linker does. However, you can't really automate this because object files might have external dependencies and the linker needs to know what these dependencies are.

> you can't really automate this because object files might have external dependencies and the linker needs to know what these dependencies are

If it can be done at runtime of the program, I guess it can be done at runtime of the linker, doesn't it?

Re: Mold: A Modern Linker

#60

Post by teddyh »

rui314 wrote: post #52
teddyh wrote: post #42

I am not sure that multi-threaded is the way to optimize file writes; I would have thought that leaving as much as possible up to the kernel, by using writev(2), splice(2), etc. would be best.

I chose mmap based on benchmarking. Writing a 2 GiB memory buffer to a file using write(2) was slower than directly constructing file contents to mmap'ed memory region. What is more interesting is the real bottleneck was not about mmap vs write(v) but in the filesystem. If you create a fresh file and write 2 GiB of data to that file, it takes like 700 milliseconds on my machine (ext4 fs), but if you write the same am…

> So, the filesystem's performance to allocate new disk blocks seems to limit the performance of my linker.

If you know the total size ahead of time, you might try using fallocate(2).

Post reply on HN