Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Making Rust binaries smaller by default

kobzol.github.io

Post reply on HN
Print view View original

141–150 of 199 posts

Re: Making Rust binaries smaller by default

#141

Post by timeon »

0x000xca0xfe wrote: post #134
lifthrasiir wrote: post #103

Earlier quoted context omitted.

But why? I mean, I'm also obsessed of the byte size from time to time and I do often optimize for the size when it's doable, but in practice anything below 1 MB seems small enough that you don't need to optimize further. There are so many low-hanging fruits when it comes to the Rust binary size...

I work in embedded software and we fight for every megabyte of NAND flash. Multiple smaller executables add up, too. Reducing the binary size is usually more important than performance (and often more important than memory safety, if we are totally honest...). Personally, I hate bloated software just as much as slow software. It's crazy what you can do with 64 kilobytes, let the demoscene blow your mind: https://www.…

> I work in embedded software

But embedded Rust usually does not use stdlib. This thread started with:

> It's really a shame that Rust includes the stdlib piecemeal in binary form, debug symbols and all, in every final binary.

Which is not case for embedded.

Re: Making Rust binaries smaller by default

#142

Post by 0x000xca0xfe »

timeon wrote: post #141
0x000xca0xfe wrote: post #134

Earlier quoted context omitted.

I work in embedded software and we fight for every megabyte of NAND flash. Multiple smaller executables add up, too. Reducing the binary size is usually more important than performance (and often more important than memory safety, if we are totally honest...). Personally, I hate bloated software just as much as slow software. It's crazy what you can do with 64 kilobytes, let the demoscene blow your mind: https://www.…

> I work in embedded software But embedded Rust usually does not use stdlib. This thread started with: > It's really a shame that Rust includes the stdlib piecemeal in binary form, debug symbols and all, in every final binary. Which is not case for embedded.

We are shipping a full blown custom Linux distribution, embedded software is more than just microcontrollers. Binary size is still just as relevant.

Re: Making Rust binaries smaller by default

#143

Post by LoganDark »

timeon wrote: post #141
0x000xca0xfe wrote: post #134

Earlier quoted context omitted.

I work in embedded software and we fight for every megabyte of NAND flash. Multiple smaller executables add up, too. Reducing the binary size is usually more important than performance (and often more important than memory safety, if we are totally honest...). Personally, I hate bloated software just as much as slow software. It's crazy what you can do with 64 kilobytes, let the demoscene blow your mind: https://www.…

> I work in embedded software But embedded Rust usually does not use stdlib. This thread started with: > It's really a shame that Rust includes the stdlib piecemeal in binary form, debug symbols and all, in every final binary. Which is not case for embedded.

Are you confusing "embedded" for "microcontrollers without an operating system"? There are plenty of embedded CPUs that can run Linux. Buildroot even exists to make it easier to create distributions for these types of devices. For example, I would consider the reMarkable 2 to be an embedded device, but it runs full-blown Linux (to the point of having a bloated Qt stack).

That's probably an extreme case though, as nobody in their right mind would actually try to run Qt on an embedded device with any reasonable limit on resources. I'm sure rM only does it because they can afford to be wasteful - they have hundreds of megabytes of space for the operating system, and all the latency-sensitive stuff is definitely not built on Qt.

Reason I know this is because they offer SSH access to the device and an SDK.

Re: Making Rust binaries smaller by default

#144

Post by 0x000xca0xfe »

MrJohz wrote: post #139
0x000xca0xfe wrote: post #134

Earlier quoted context omitted.

I work in embedded software and we fight for every megabyte of NAND flash. Multiple smaller executables add up, too. Reducing the binary size is usually more important than performance (and often more important than memory safety, if we are totally honest...). Personally, I hate bloated software just as much as slow software. It's crazy what you can do with 64 kilobytes, let the demoscene blow your mind: https://www.…

In fairness, this is just doing something automatically that you could have done manually anyway. It's a good change because defaults matter, but it's always been possible to strip debug symbols from Rustc-generated binaries, and all the guides I've seen to reducing binary stuff have mentioned this already. It's also important to remember that in embedded software, you're probably working in a no_std environment anyw…

I'm not criticizing Rust, I'm criticizing statements that doubt whether the effort to reduce the binary size is worth it.

We currently have some nasty C++/boost monstrosities, if Rust can deliver a better development experience AND reliable stack traces AND smaller binaries, it would be HUGE for embedded.

Re: Making Rust binaries smaller by default

#145

Post by ckok »

josephg wrote: post #111
pdimitar wrote: post #100

It's really a shame that Rust includes the stdlib piecemeal in binary form, debug symbols and all, in every final binary. I do love Rust but binary sizes have always annoyed me greatly and I always had this nagging feeling that part of all programmers don't take Rust seriously because of it. And I actually have witnessed, several times in the last 2-ish years, older-school programmers berating and ignoring Rust on th…

> (and not just blindly copy-pasting the 4MB binary blob) Where is this 4mb claim coming from? I just built a hello world on macos in release mode, with no special flags and the result was 400kb. Thats absolutely larger that it should be, but its a lot smaller than 4mb. Is it really that much worse on linux?

Macho executables don't contain debug info. Just references to .o files.

Re: Making Rust binaries smaller by default

#146

Post by lifthrasiir »

0x000xca0xfe wrote: post #134
lifthrasiir wrote: post #103

Earlier quoted context omitted.

But why? I mean, I'm also obsessed of the byte size from time to time and I do often optimize for the size when it's doable, but in practice anything below 1 MB seems small enough that you don't need to optimize further. There are so many low-hanging fruits when it comes to the Rust binary size...

I work in embedded software and we fight for every megabyte of NAND flash. Multiple smaller executables add up, too. Reducing the binary size is usually more important than performance (and often more important than memory safety, if we are totally honest...). Personally, I hate bloated software just as much as slow software. It's crazy what you can do with 64 kilobytes, let the demoscene blow your mind: https://www.…

While I'm not a demoscener, I know enough about how they are generally made and have my own share of award-winning small programs. So I'm confident that they don't make a good example of non-bloated software.

Every demo is really amazing when you encounter them for the first time, but AFAIK 64K demos were hottest around 2000 when Farbrausch revolutionalized the scene with .fr-08: .the .product [1]. Since then, 64K turned out to be too large because most 64K demos can be divided into multiple parts---engine, data and compressor---and each part can be individually developed. There are many 64K demos but far less engines and only a handful number of compressors in this level. It also means that there are only a handful number of people that can actually make engines and compressors. It's not a fit software, it's rather an unhealthily thin software. They are still awesome but they can't be a model.

[1] http://www.theproduct.de/

Re: Making Rust binaries smaller by default

#147

Post by ponkpanda »

josephg wrote: post #104
rfoo wrote: post #58

Earlier quoted context omitted.

opt-level = "z" may be slightly slower than O2/O3. lto = true and lto-units = 1 makes the final linking unbearably slow for large programs. These are still okay, but then `panic = "abort"` shakes a lot of code off by making the final executable unable to print a nice stack trace (even without symbols) and instead immediately traps/abort()-s. Edit: and GP also used "panic-immediate-abort", which also removes the depen…

> These are still okay, but then `panic = "abort"` shakes a lot of code off by making the final executable unable to print a nice stack trace (even without symbols) and instead immediately traps/abort()-s. I just tried this. The stack traces on panic seem more or less the same with or without panic = "abort" in Cargo.toml. For example, this program: fn main() { let v = vec![1, 2, 3]; v[99]; } Compiled with panic="abo…

You're not adding the +nightly switch and the required flags to compiled std. Just setting panic=abort in cargo.toml isn't enough.

One can get binaries pretty damn small (low-mid tens of kilobytes for a basic cli program doing something like hashing of a file).

Problem I've found with manually compiling std (which has ancillary benefits of being able to compile to a specific uarch) is it can break the compilation process when bringing in third-party deps. The config.toml (stored in $PROJECT_ROOT/.cargo) overrides cargo's behaviour for all dependencies as well - which may break those compilations.

Tbh, it's one reason I don't particularly rate the rustc+cargo toolchain - but for most people writing regular applications: just being able to do ```cargo build -r``` and not care about binary size, uarch optimization or custom llvm/rustc optimizations (PGO etc), most won't care.

Re: Making Rust binaries smaller by default

#148

Post by tialaramex »

lifthrasiir wrote: post #118
bipson wrote: post #107

Earlier quoted context omitted.

Depends on what you're aiming for. If we talk about ultra-low-power platforms, e.g. energy-harvesting IoT devices, 1MB is still quite a lot. If we are going to argue that Rust can compete with C/C++, it needs to have similar performance, also regarding binary size.

C/C++ standard library doesn't work in that environment anyway---you typically need a freestanding mode. Rust equvialent is `#[no_std]` which works well for popular boards.

To be technical, Rust deliberately splits out three elements: core (which is necessary for the Rust language and is always provided) alloc (which needs a heap allocator, not applicable if you don't have or don't want one) and then std (which expects an operating system, with features like files and sockets and threads and knowing what the time is)

C++ just labels a few specific features as available in a "freestanding" C++ standard library if you have one (on an embedded platform presumably you do).

This makes it very easy to know what you're getting in Rust's stdlib in #[no_std] because it's all of core, so e.g.

https://doc.rust-lang.org/core/primitive.slice.html#method.s... vs. https://doc.rust-lang.org/std/primitive.slice.html#method.so...

At first glance those are identical, but no, std is re-exporting every feature core had, but it also gains features, for example the stable sort function family only exist in std because they're using a temporary allocation whereas the unstable (ie equivalent items may be re-ordered) sort provided even in core doesn't do that.

Figuring out what you get in your stdlib with C++ often comes down to suck it and see.

Re: Making Rust binaries smaller by default

#149

Post by kobzol »

weinzierl wrote: post #86

I was wondering if dead code elimination also removes the debug symbols related to the dead code? The ~ 4 MiB of debug symbols the article talks about are for the whole libstd or just the portions that actually ended up in the binary? I think the new default makes sense but I'd love to have the option to build a lean but debuggable release binary with just the needed symbols.

You can't really use dead code elimination on debug symbols, because you don't know which symbols will you need. You would need to know where and how will your program crash or if the user will want to use a debugger on it.

Re: Making Rust binaries smaller by default

#150

Post by pdimitar »

nindalf wrote: post #108
pdimitar wrote: post #100

It's really a shame that Rust includes the stdlib piecemeal in binary form, debug symbols and all, in every final binary. I do love Rust but binary sizes have always annoyed me greatly and I always had this nagging feeling that part of all programmers don't take Rust seriously because of it. And I actually have witnessed, several times in the last 2-ish years, older-school programmers berating and ignoring Rust on th…

> so I guess I'll have to wait several more years A feature that's landed in Rust nightly will be part of the next beta release (at most 6 weeks away) and then the following full release (exactly 6 weeks away). For this feature in particular, rustbot added a tag of 1.77.0, which is releasing on 21st March 2024, less than 2 months away. It's possible you've confused this with more complex features that stay on nightly…

I indeed am confusing it with those features that seem to be in nightly forever. Thank you, compiling with beta is much more acceptable for me. (EDIT: or simply waiting for stable 1.77)
Post reply on HN