Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Wild – A fast linker for Linux

github.com

Post reply on HN
Print view View original

21–30 of 222 posts

Re: Wild – A fast linker for Linux

#21

Post by Philpax »

ComputerGuru wrote: post #16
senkora wrote: post #9

Earlier quoted context omitted.

I assume that he is referring to "fearless concurrency", the idea that Rust makes it possible to write more complex concurrent programs than other languages because of the safety guarantees: https://doc.rust-lang.org/book/ch16-00-concurrency.html So the logic would go: 1. mold doesn't do incremental linking because it is too complex to do it while still being fast (concurrent). 2. Rust makes it possible to write very…

Actually a lot of the hacks that mold uses to be the fastest linker would be, ironically, harder to reproduce with rust because they’re antithetical to its approach. Eg Mold intentionally eschews used resource collection to speed up execution (it’ll be cleaned up by the os when the process exits) while rust has a strong RAII approach here that would introduce slowdowns.

I mean, that's pretty easy to do in Rust: https://doc.rust-lang.org/std/mem/struct.ManuallyDrop.html

Also see various arena allocator crates, etc.

Re: Wild – A fast linker for Linux

#22

Post by IshKebab »

devit wrote: post #7

I think the optimal approach for development would be to not produce a traditional linked executable at all, but instead just place the object files in memory, and then produce a loader executable that hooks page faults in those memory areas and on-demand mmaps the relevant object elsewhere, applies relocations to it, and then moves it in place with mremap. Symbols would be resolved based on an index where only updat…

Sounds like dynamic linking, sort of.

Re: Wild – A fast linker for Linux

#23

Post by manoweb »

throwaway106382 wrote: post #4

> Mold is already very fast, however it doesn't do incremental linking and the author has stated that they don't intend to. Wild doesn't do incremental linking yet, but that is the end-goal. By writing Wild in Rust, it's hoped that the complexity of incremental linking will be achievable. Can someone explain what is so special about Rust for this?

That is baffling. Maybe the author assumes that a language with many safeguards will lead to keeping complexity under control for a difficult task.

By the way I had to lookup what incremental linking is, in practice I think it means that code from libraries and modules that have not changed won’t need to be re-packed each time which ch will save time for frequent development builds, it’s actually ingenious

Re: Wild – A fast linker for Linux

#24

Post by [deleted] »

throwaway106382 wrote: post #4

> Mold is already very fast, however it doesn't do incremental linking and the author has stated that they don't intend to. Wild doesn't do incremental linking yet, but that is the end-goal. By writing Wild in Rust, it's hoped that the complexity of incremental linking will be achievable. Can someone explain what is so special about Rust for this?

[deleted]

Re: Wild – A fast linker for Linux

#25

Post by cogman10 »

ComputerGuru wrote: post #16
senkora wrote: post #9

Earlier quoted context omitted.

I assume that he is referring to "fearless concurrency", the idea that Rust makes it possible to write more complex concurrent programs than other languages because of the safety guarantees: https://doc.rust-lang.org/book/ch16-00-concurrency.html So the logic would go: 1. mold doesn't do incremental linking because it is too complex to do it while still being fast (concurrent). 2. Rust makes it possible to write very…

Actually a lot of the hacks that mold uses to be the fastest linker would be, ironically, harder to reproduce with rust because they’re antithetical to its approach. Eg Mold intentionally eschews used resource collection to speed up execution (it’ll be cleaned up by the os when the process exits) while rust has a strong RAII approach here that would introduce slowdowns.

Depends on how things are approached.

You could, for example, take advantage of bump arena allocator in rust which would allow the linker to have just 1 alloc/dealloc. Mold is still using more traditional allocators under the covers which won't be as fast as a bump allocator. (Nothing would stop mold from doing the same).

Re: Wild – A fast linker for Linux

#26

Post by ComputerGuru »

Philpax wrote: post #21
ComputerGuru wrote: post #16

Earlier quoted context omitted.

Actually a lot of the hacks that mold uses to be the fastest linker would be, ironically, harder to reproduce with rust because they’re antithetical to its approach. Eg Mold intentionally eschews used resource collection to speed up execution (it’ll be cleaned up by the os when the process exits) while rust has a strong RAII approach here that would introduce slowdowns.

I mean, that's pretty easy to do in Rust: https://doc.rust-lang.org/std/mem/struct.ManuallyDrop.html Also see various arena allocator crates, etc.

Not really. You would have to either wrap any standard library types in newtypes with ManuallyDrop implemented or (for some) use a custom allocator. And if you want to free some things in one go but not others that gets much harder, especially when you look at how easy a language like zig makes it.

And if you intentionally leak everything it is onerous to get the borrow checker to realize that unless you use a leaked box for all declaration/allocations, which introduces both friction and performance regressions (due to memory access patterns) because the use of custom allocators doesn’t factor into lifetime analysis.

(Spoken as a die-hard rust dev that still thinks it’s the better language than zig for most everything.)

Re: Wild – A fast linker for Linux

#27

Post by secondcoming »

pzmarzly wrote: post #19

Ever since mold relicensed from AGPL to MIT (as part of mold 2.0 release), the worldwide need for making another fast linker has been greatly reduced, so I wasn't expecting a project like this to appear. And definitely wasn't expecting it to already be 2x faster than mold in some cases. Will keep an eye on this project to see how it evolves, best of luck to the author.

Maybe I'm holding it wrong, but mold isn't faster at all if you're using LTO, which you probably should be.

Re: Wild – A fast linker for Linux

#30

Post by fsfod »

devit wrote: post #7

I think the optimal approach for development would be to not produce a traditional linked executable at all, but instead just place the object files in memory, and then produce a loader executable that hooks page faults in those memory areas and on-demand mmaps the relevant object elsewhere, applies relocations to it, and then moves it in place with mremap. Symbols would be resolved based on an index where only updat…

You can sort of do that with some of LLVM's JIT systems https://llvm.org/docs/JITLink.html, I'm surprised that no one has yet made a edit and continue system using it.
Post reply on HN