Wild – A fast linker for Linux • Hacker News Board

Re: Wild – A fast linker for Linux

#11

Post by devit » Fri, Jan 24, 2025, 4:52 PM UTC

throwaway106382 wrote: post #4

> Mold is already very fast, however it doesn't do incremental linking and the author has stated that they don't intend to. Wild doesn't do incremental linking yet, but that is the end-goal. By writing Wild in Rust, it's hoped that the complexity of incremental linking will be achievable. Can someone explain what is so special about Rust for this?

It's feasible to write complex correct programs with optimal performance in Rust, unlike any other programming language (complex+correct is not feasible in C/C++/assembly/Zig/etc., optimal performance not possible in any other language).

Re: Wild – A fast linker for Linux

#12

Post by satvikpendem » Fri, Jan 24, 2025, 4:52 PM UTC

I looked at this before, is it ready for production? I thought not based on the readme, so I'm still using mold.

For those on macOS, Apple released a new linker about a year or two ago (which is why the mold author stopped working on their macOS version), and if you're using it with Rust, put this in your config.toml:

    [target.aarch64-apple-darwin]
    rustflags = [ 
        "-C",
        "link-arg=-fuse-ld=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld",
        "-C",
        "link-arg=-ld_new",
    ]

Re: Wild – A fast linker for Linux

#13

Post by ComputerGuru » Fri, Jan 24, 2025, 4:52 PM UTC

There’s been a lot of interest in faster linkers spurred by the adoption and popularity of rust.

Even modest statically linked rust binaries can take a couple of minutes in the link stage of compilation in release mode (using mold). It’s not a rust-specific issue but an amalgam of (usually) strictly static linking, advanced link-time optimizations enabled by llvm like LTO and bolt, and a general dissatisfaction with compile times in the rust community. Rust’s (clinically) strong relationship with(read: dependency on) LLVM makes it the most popular language where LLVM link-time magic has been most heavily universally adopted; you could face these issues with C++ but it wouldn’t be chalked up to the language rather than your toolchain.

I’ve been eyeing wild for some time as I’m excited by the promise of an optimizing incremental linker, but to be frank, see zero incentive to even fiddle with it until it can actually, you know, link incrementally.

Re: Wild – A fast linker for Linux

#14

Post by compiler-guy » Fri, Jan 24, 2025, 4:53 PM UTC

throwaway106382 wrote: post #4

> Mold is already very fast, however it doesn't do incremental linking and the author has stated that they don't intend to. Wild doesn't do incremental linking yet, but that is the end-goal. By writing Wild in Rust, it's hoped that the complexity of incremental linking will be achievable. Can someone explain what is so special about Rust for this?

That’s puzzling to me too. Rust is a great language, and probably makes developing Wild faster. But the complexity of incremental linking doesn’t stem from the linker’s implementation language. It stems from all the tracking, reserved spacing, and other issues required to link a previously linked binary (or at least parts of it) a second time.

Re: Wild – A fast linker for Linux

#15

Post by compiler-guy » Fri, Jan 24, 2025, 4:54 PM UTC

senkora wrote: post #9

throwaway106382 wrote: post #4

> Mold is already very fast, however it doesn't do incremental linking and the author has stated that they don't intend to. Wild doesn't do incremental linking yet, but that is the end-goal. By writing Wild in Rust, it's hoped that the complexity of incremental linking will be achievable. Can someone explain what is so special about Rust for this?

I assume that he is referring to "fearless concurrency", the idea that Rust makes it possible to write more complex concurrent programs than other languages because of the safety guarantees: https://doc.rust-lang.org/book/ch16-00-concurrency.html So the logic would go: 1. mold doesn't do incremental linking because it is too complex to do it while still being fast (concurrent). 2. Rust makes it possible to write very…

Both mold and lld are already very heavily concurrent. There is no fear at all there.

Re: Wild – A fast linker for Linux

#16

Post by ComputerGuru » Fri, Jan 24, 2025, 4:54 PM UTC

senkora wrote: post #9

throwaway106382 wrote: post #4

> Mold is already very fast, however it doesn't do incremental linking and the author has stated that they don't intend to. Wild doesn't do incremental linking yet, but that is the end-goal. By writing Wild in Rust, it's hoped that the complexity of incremental linking will be achievable. Can someone explain what is so special about Rust for this?

I assume that he is referring to "fearless concurrency", the idea that Rust makes it possible to write more complex concurrent programs than other languages because of the safety guarantees: https://doc.rust-lang.org/book/ch16-00-concurrency.html So the logic would go: 1. mold doesn't do incremental linking because it is too complex to do it while still being fast (concurrent). 2. Rust makes it possible to write very…

Actually a lot of the hacks that mold uses to be the fastest linker would be, ironically, harder to reproduce with rust because they’re antithetical to its approach. Eg Mold intentionally eschews used resource collection to speed up execution (it’ll be cleaned up by the os when the process exits) while rust has a strong RAII approach here that would introduce slowdowns.

Re: Wild – A fast linker for Linux

#17

Post by brink » Fri, Jan 24, 2025, 4:55 PM UTC

satvikpendem wrote: post #12

I looked at this before, is it ready for production? I thought not based on the readme, so I'm still using mold. For those on macOS, Apple released a new linker about a year or two ago (which is why the mold author stopped working on their macOS version), and if you're using it with Rust, put this in your config.toml: [target.aarch64-apple-darwin] rustflags = [ "-C", "link-arg=-fuse-ld=/Applications/Xcode.app/Content…

I don't even use mold for production. It's for development.

Re: Wild – A fast linker for Linux

#18

Post by compiler-guy » Fri, Jan 24, 2025, 4:55 PM UTC

dralley wrote: post #6

throwaway106382 wrote: post #4

> Mold is already very fast, however it doesn't do incremental linking and the author has stated that they don't intend to. Wild doesn't do incremental linking yet, but that is the end-goal. By writing Wild in Rust, it's hoped that the complexity of incremental linking will be achievable. Can someone explain what is so special about Rust for this?

I assume they're referring to thread-safety and the ability to more aggressively parallelize.

Mold and lld are already very heavily parallelized. It’s one of the things that makes them very fast already.

Re: Wild – A fast linker for Linux

#19

Post by pzmarzly » Fri, Jan 24, 2025, 4:55 PM UTC

Ever since mold relicensed from AGPL to MIT (as part of mold 2.0 release), the worldwide need for making another fast linker has been greatly reduced, so I wasn't expecting a project like this to appear. And definitely wasn't expecting it to already be 2x faster than mold in some cases. Will keep an eye on this project to see how it evolves, best of luck to the author.

Re: Wild – A fast linker for Linux

#20

Post by IshKebab » Fri, Jan 24, 2025, 4:57 PM UTC

throwaway106382 wrote: post #4

> Mold is already very fast, however it doesn't do incremental linking and the author has stated that they don't intend to. Wild doesn't do incremental linking yet, but that is the end-goal. By writing Wild in Rust, it's hoped that the complexity of incremental linking will be achievable. Can someone explain what is so special about Rust for this?

There are two main factors:

1. Rust's well designed type system and borrow checker makes writing code that works just easier. It has the "if it compiles it works" property (not unique to Rust; people say this about e.g. Haskell too).

2. Rust's type system - especially its trait system can be used to enforce safety constraints statically. The obvious one is the Send and Sync traits for thread safety, but there are others, e.g. the Fuchsia network code statically guarantees deadlocks are impossible.

Mold is written in C++ which is extremely error prone in comparison.