Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Linux Kernel Prepares for Rust 1.77 Upgrade

phoronix.com

Post reply on HN
Print view View original

61–70 of 107 posts

Re: The Linux Kernel Prepares for Rust 1.77 Upgrade

#61

Post by habibur »

Narishma wrote: post #33
habibur wrote: post #26

Earlier quoted context omitted.

You can strip C compiled binaries too. And that halves the binary size. The point is for example a hello world Rust binaries is 300kb after striping while C compiled one is 15kb. A difference of 20 times.

But C doesn't statically link the standard library by default like Rust does.

Hello world deps for C :

    linux-vdso.so.1 (0x00007fff25cb8000)
    libc.so.6 => /lib64/libc.so.6 (0x00007fe5f08d9000)
    /lib64/ld-linux-x86-64.so.2 (0x00007fe5f0ae2000)
And for Rust

    linux-vdso.so.1 (0x00007ffc109f9000)
    libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f8eda404000)
    libc.so.6 => /lib64/libc.so.6 (0x00007f8eda222000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f8eda4a8000)
Rather Rust has 1 more dynamically linked library than C.

Re: The Linux Kernel Prepares for Rust 1.77 Upgrade

#62

Post by tialaramex »

paavohtl wrote: post #51
almatabata wrote: post #44

Earlier quoted context omitted.

If you ignore dependencies and stick to stable features yes. If you include dependencies then it can happen that a dependency relies on unstable features. In which case you might have to upgrade the library version (if they support the new compiler version). The library might have changed the API by then which would force you to change your code. Except for the above use case, upgrades to the latest version of the co…

Dependencies are not allowed to use unstable features either with the stable compiler. The only exception is the standard library, which uses numerous unstable features even with a stable distribution of Rust.

Worth briefly explaining the rationale for this (stdlib gets to use unstable features)

Rust's stdlib is maintained with the rest of the language and by the same broad team, so, if you're tweaking unstable feature X, you are also responsible for ensuring the stdlib people using feature X sort that out. I'm not sure if Rust's internal policies mean you shouldn't land a change to the main tree without accompanying stdlib patches, or whether you're only required to give them adequate notice, but either way it's not going out the door in a stable release being incompatible with its own implementation.

This couldn't really work with 3rd party libraries.

Re: The Linux Kernel Prepares for Rust 1.77 Upgrade

#63

Post by fl0ki »

nequo wrote: post #42
pornel wrote: post #36

Earlier quoted context omitted.

Not quite. Every Rust program will have some code path that may panic, and the default panic handler uses debug formatting, which uses dynamic dispatch, which prevents elimination of the rest of the printing machinery. There’s panic_immediate_abort unstable setting that makes Rust panics crash as hard as a C segfault, and only then you can get rid of a good chunk of stdlib.

Does this mean that only the printing machinery is not eliminated or that other parts of stdlib are present in the binary too even though unused?

The printing machinery alone is quite large when you consider that it includes the code & raw data for Unicode, whether or not similar facilities were already available on the host libc. Though you're not likely to avoid that in any non-trivial Rust program anyway, as even a pretty barebones CLI will need Unicode-aware string processing.

I generally find Rust binaries to be "a few" megabytes if they don't have an async runtime, and a few more if they do. It has never bothered me on an individual program basis, but I can imagine it adding up over an entire distribution with hundreds of individual binaries. I see the very real concern there, but personally I would still not risk ABI hazards just to save on space.

Re: The Linux Kernel Prepares for Rust 1.77 Upgrade

#64

Post by fl0ki »

tialaramex wrote: post #52
surajrmal wrote: post #37

Earlier quoted context omitted.

Overuse of monorphisation by the community is typically the right choice given the average use case for them is servers where this need not make a meaningful difference. As with any ecosystem, choices folks make may not be suitable for everyone. Those differing requirements require fracturing into smaller ecosystems which share common requirements. Ultimately, that's what's happening with rust and it's very healthy t…

I believe the menu of options is undesirable until you actually know you have requirements you can evaluate them against. As much as possible even if I do have a choice, there should be a default and I needn't be asked. When I make a Rust project, cargo notices I have git and, since I didn't say otherwise, it mints a Git repo for the new project automatically. It doesn't insist on asking if I want one, and then askin…

I agree, but I have yet to see a single real-world example of a Rust project meaningfully reducing its binary size by switching from monomorphization to dynamic dispatch in its own code. Many Rust developers boast that they virtually never use `dyn`, but then still appeal to it when arguing that Rust has dynamic dispatch so monomorphization is an avoidable cost.

Sometimes you can provide `T = Arc/Box` where `T: Foo` is required, but only if the trait is designed to be object-safe, not simply by default. If you get to design the trait and all of its consumers yourself, you might have this option, but it's very possible that you're using a library that does not make this possible. You can easily be the first person to bother trying the `dyn` for a trait and running into these limitations.

Besides that, you might not even have that much control of the concrete type used. For example, if you are generating large schemas with serde, serde decides how that code is monomorphized, not you. In contrast, for better or worse, the path of least resistance in Go is to use a reflection-based serialization framework which has notable runtime costs (that may or may not matter to a given project) but successfully avoids compile time and binary size costs. (There are other reasons that Go binaries end up even larger than Rust ones, this just isn't one of them)

Despite Rust's general principle of giving its users informed choices here, I am not aware of any option that does 100% dynamic dispatch for (de)serialization, so in practice this is a largely unavoidable cost in each project that is decided only by how complex the schema is.

It's also only fair to point out that C++ tends to end up in this place too, mitigated only by dynamic linking and not any magical property of the language itself. Even C can head this way because monomorphizing with macros has the same effect, though due to how such code is structured, it's also less likely to be inlined than C++ or Rust.

Re: The Linux Kernel Prepares for Rust 1.77 Upgrade

#65

Post by jcranmer »

tialaramex wrote: post #62
paavohtl wrote: post #51

Earlier quoted context omitted.

Dependencies are not allowed to use unstable features either with the stable compiler. The only exception is the standard library, which uses numerous unstable features even with a stable distribution of Rust.

Worth briefly explaining the rationale for this (stdlib gets to use unstable features) Rust's stdlib is maintained with the rest of the language and by the same broad team, so, if you're tweaking unstable feature X, you are also responsible for ensuring the stdlib people using feature X sort that out. I'm not sure if Rust's internal policies mean you shouldn't land a change to the main tree without accompanying stdli…

There's a second category of unstable features to mention here.

Some of the features are essentially perma-unstable, because they're exposing some compiler intrinsics for the library to be able to use. This is the equivalent of things like __builtin_* for C compilers.

Re: The Linux Kernel Prepares for Rust 1.77 Upgrade

#66

Post by fl0ki »

habibur wrote: post #61
Narishma wrote: post #33

Earlier quoted context omitted.

But C doesn't statically link the standard library by default like Rust does.

Hello world deps for C : linux-vdso.so.1 (0x00007fff25cb8000) libc.so.6 => /lib64/libc.so.6 (0x00007fe5f08d9000) /lib64/ld-linux-x86-64.so.2 (0x00007fe5f0ae2000) And for Rust linux-vdso.so.1 (0x00007ffc109f9000) libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f8eda404000) libc.so.6 => /lib64/libc.so.6 (0x00007f8eda222000) /lib64/ld-linux-x86-64.so.2 (0x00007f8eda4a8000) Rather Rust has 1 more dynamically linked library…

That might be true for Hello World, but libgcc_s is where a lot of builtins for C itself go, so you'll find it ends up linked into a lot of non-trivial C programs as well. See https://gcc.gnu.org/onlinedocs/gccint/Libgcc.html

Re: The Linux Kernel Prepares for Rust 1.77 Upgrade

#67

Post by fl0ki »

surajrmal wrote: post #39
CUViper wrote: post #27

Earlier quoted context omitted.

Rust is backwards compatible when you stick to stable features, but the kernel uses unstable features that can and do incur breaking changes. https://github.com/Rust-for-Linux/linux/issues/2

It seems prudent to limit rust usage in the kernel until that list can be burned down to zero. It makes sense that you need to at least get rust in the kernel to find out what missing features you need to have implemented and stabilized, but excessive use will make folks lives painful as they try to track upstream rust releases.

Please bear in mind that Linux has used non-standard GCC extensions to C for decades as well. The tradeoffs here are their call to make.

Besides, at this stage, it makes perfect sense for Linux to use unstable Rust features. It was one thing to say Rust should be great for writing kernels, it's another to actually get feedback on how it needs to be better, and that's only possible if the potential improvements are motivated by those who need them and incubated without the constraints of backwards compatibility nor the risks of locking in permanent tech debt.

Rust's unstable feature concept was designed for exactly this kind of freeform evolution and it's working exactly as intended. As for the specific tradeoffs being made in Linux, its contributors are in a much better position to weigh those than we are.

Re: The Linux Kernel Prepares for Rust 1.77 Upgrade

#68

Post by vlovich123 »

makapuf wrote: post #7
kramerger wrote: post #4

The LKM post mentions binary size improvements. One issues I have had with Rust applications is the huge binary size (yes, I know this has improved a bit lately). Is there a good comparison between kernel C and kernel Rust code in this regard?

From (1) I guess most of the binsize comes from stdlib that the kernel does not use, so I guess that's two different problems. (1) https://github.com/johnthagen/min-sized-rust

That wouldn’t explain what the size improvements are from the upgrade as the kernel always built nostd.

Re: The Linux Kernel Prepares for Rust 1.77 Upgrade

#69

Post by forrestthewoods »

kramerger wrote: post #4

The LKM post mentions binary size improvements. One issues I have had with Rust applications is the huge binary size (yes, I know this has improved a bit lately). Is there a good comparison between kernel C and kernel Rust code in this regard?

> is the huge binary size

Can you quantify this? How big is too big? Ideally for a real program, and not an experiment to make the tiniest possible program.

On my Windows machine ripgrep rg.exe is just 4.2mb. Making that smaller feels irrelevant.

I’m not convinced that binary size is a real problem. But I’m open to evidence!

Re: The Linux Kernel Prepares for Rust 1.77 Upgrade

#70

Post by ericbarrett »

bluejekyll wrote: post #17
kramerger wrote: post #4

The LKM post mentions binary size improvements. One issues I have had with Rust applications is the huge binary size (yes, I know this has improved a bit lately). Is there a good comparison between kernel C and kernel Rust code in this regard?

This is a good guide on building small Rust binaries: https://github.com/johnthagen/min-sized-rust This talks about going to extreme lengths on making the smallest Rust binary possible, 400 bytes when it was written, https://darkcoding.net/software/a-very-small-rust-binary-ind... The thing is, you lose a lot of nice features when you do this, like panic unwinding, debug symbols, stdlib… for kernel and some embedded d…

> debug symbols

In ye olden days it was common to distribute a binary without debug symbols, but to keep a copy of them for every released build¹. If an application crashed (panicked, signalled, etc.) you got a core dump that you could debug using the stripped binary together with the symbol file. This gave you both smaller binary sizes and full debugging capability at the cost of some extra administration. I'm not sure if this is possible with "stock" Rust, but if you need lean binaries but want to do forensic investigation it's something to look into.

1. https://sourceware.org/gdb/current/onlinedocs/gdb.html/Separ...

Post reply on HN