Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rust in the kernel is no longer experimental

lwn.net

Post reply on HN
Print view View original

781–790 of 853 posts

Re: Rust in the kernel is no longer experimental

#781

Post by menaerus »

aw1621107 wrote: post #730
menaerus wrote: post #707

Earlier quoted context omitted.

Yes, but my point is when things blow up how exactly do you know which unsafe block you should look into? From their statement it appears as if there's such a simple correlation between "here's your segfault" and "here's your unsafe block that caused it", and which I believe there isn't, and which is why I said there's no encapsulation, local reasoning etc.

> Yes, but my point is when things blow up how exactly do you know which unsafe block you should look into? In the most general case, you don't. But again, I think that that rather misses the point the statement was trying to get at. Perhaps a more useful framing for you would be that in the most general case the encapsulation and local reasoning here is between modules that use unsafe and everything else. In some (m…

> in the most general case the encapsulation and local reasoning here is between modules that use unsafe and everything else

This would be the same narrative as in, let's say, C++. Wrap the difficult and low-level memory juggling stuff into "modules", harden the API, return the references and/or smart-pointers, and then just deal with the rest of the code with ease, right? Theoretically possible but practically impossible.

First reason is that abstractions get really leaky, and they especially get really leaky in the code that demands the upmost performance. Anyone who implemented their own domain/workload specific hash-map or mutex or anything similarly foundational will understand this sentiment. Anyway, if we just have a look into the NVMe driver above, there're no "unsafe modules".

Second, and as I already argued, UB in the module library transcends into the rest of your code so I fail to understand how is it so that the dozens of unsafe sections make the reasoning or debugging any more simpler when reasoning is actually not a function of number of unsafe sections but it is the function of interactions between different parts of the code that end up touching the memory in the unsafe block in a way that it was not anticipated. This is almost always the case when dealing with undefined behavior.

> I don't get that sense from the statement at all.

It is a bit exaggerated example of mine but I do - their framing suggests ~exactly that and which is simply not true.

Re: Rust in the kernel is no longer experimental

#782

Post by littlestymaar »

menaerus wrote: post #778
littlestymaar wrote: post #711

Earlier quoted context omitted.

> especially in such a complex codebase You accidentally put the finger on the key point, emphasis mine. When you have a memory-unsafe language, the complexity of the whole codebase impact your ability to uphold memory-related invariants. But unsafe block are, by definition, limited in scope and assuming you design your codebase properly, they shouldn't interact with other unsafe blocks in a different module. So the…

No, this is just an example of confirmation bias. You're given a totally unrealistic figure of 1 vuln per 200K/5M LoC and now you're hypothesizing why that could be so. Google, for anyone unbiased, lost the credibility when they put this figure into the report. I wonder what was their incentive for doing so. > But unsafe block are, by definition, limited in scope and assuming you design your codebase properly, they s…

> You're given a totally unrealistic figure of 1 vuln per 200K/5M LoC and now you're hypothesizing why that could be so.

You are the one claiming it's unrealistic. And you gave zero argument why besides “the codebase is complex”, which I refuted. See the definition of complexity:

> The term is generally used to characterize something with many parts where those parts interact with each other in multiple ways, culminating in a higher order of emergence greater than the sum of its parts

Each unsafe block may be “difficult” in itself, but the resulting system isn't “complex” because you don't have this compounding effect.

> I wonder what was their incentive for doing so.

And obviously it must be malice…

> For anyone who has written low-level code with substantial complexity knows that this is just a wishful thinking. In such code, abstractions fall-apart and "So the complexity related to one unsafe block is in fact contained to his own module, and doesn't spread outside" is just wrong as I explained in my other comment here - UB taking place in unsafe section will transcend into the rest of the "safe" code - UB is not "caught" or put into the quarantine with some imaginative safety net at the boundary between the safe and unsafe sections.

I think you don't understand the problem as well as you think you do. Of course if the UB happens then all bets are off! Its consequences won't be limited to a part of the code, by definition. And nobody said otherwise.

But for the UB to happen, there must be some violation of an memory invariant (the most common would be using a value after free, freeing twice, accessible the same memory from multiple threads without synchronization or, and this is specific to Rust, violating reference aliasing rules).

To avoid violating these invariants, the programmer must have a mental model of the ownership over all the system on which these invariants apply. For C or C++, it means having a mental model of all the code base, because the invariants related to one piece of code can be violated from everywhere.

In Rust this is different, you're not going to have raw pointers to one piece of data being used in multiple parts of the code (well, if you really want, nobody stops you, but I'm confident the Android team didn't). And as such, you'll have to think about the invariants only at the scale of one module. Building an accurate mental model of a 350-line module is much more tractable for a human than doing the same for an entire codebase, and it's not even close.

Re: Rust in the kernel is no longer experimental

#783

Post by timeon »

andersmurphy wrote: post #769
josephg wrote: post #412

Earlier quoted context omitted.

Why? Are you allergic to running rust code? You know, you can’t actually tell when you execute a binary? I don’t like programming in Go, but nothing stops me running go programs on my computer. My computer doesn’t seem to care.

It's not about rust or no rust, rust is fine. It's about the constant drama.

> It's about the constant drama.

So why are you posting things like drama queen:

> I guess it's time to finally try FreeBSD.

(btw. FreeBSD is fine)

Re: Rust in the kernel is no longer experimental

#785

Post by jacquesm »

sunshowers wrote: post #743
jacquesm wrote: post #505

Earlier quoted context omitted.

I think the main issue is that it never was about how rust programmers should write more rust. Just like a religion it is about what other people should do. That's why you see so many abandoned very ambitious rust projects to tackle x,y or z now written in C to do them all over again (and throw away many years of hardening and bug fixes). The idea is that the original authors can be somehow manipulated into taking th…

> Rust should have done exactly one thing and do that as good as possible: be a C replacement and do that while sticking as close as possible to the C syntax. The goal of Rust is to build reliable systems software like the kind I've worked on for the last many years, not to be a better C, the original goal of which was to be portable assembler for the PDP-11. > Now we have something that is a halfway house between C,…

Haskell is interesting, it is probably one of the best programming languages ever devised but it just can't seem to gain traction. There are some isolated pockets where it does well (both business wise as well as geographically).

Re: Rust in the kernel is no longer experimental

#786

Post by delamon »

estebank wrote: post #652
delamon wrote: post #526

Earlier quoted context omitted.

Can you elaborate, what key assumptions about memory safety linked lists break? Sure, double linked lists may have non-trivial ownership, but that doesn't compromise safety.

Rust wants all memory to be modeled as an ownership tree: the same bit of memory can't be owned by more than one data structure. A doubly linked list breaks that requirement so it can't be modeled in safe Rust directly. The options are using unsafe, or using one of the pointer wrapper types that have runtime checks that ensure correct behavior and own the underlying memory as far Rust is concerned.

Right. So it is not that double-linked lists are inherently unsafe, it is (just) Rust ownership model cannot represent them (any other cyclic structures).

Re: Rust in the kernel is no longer experimental

#787

Post by [deleted] »

CursedSilicon wrote: post #765
glenstein wrote: post #645

Earlier quoted context omitted.

Thank you for asking this. It was presented matter of fact and I would not have appreciated that it was something less than settled. The best I can find from googling was rumors and speculation (insert comment in reply with more rumors and speculation). I also don't necessarily think it means someone is a Bad Guy if true. I want people to be able to have anonymous alter egos online if that's what they want and they'r…

Being called a "concern troll" for daring to question such a bizarre accusation makes me think it's just a bunch of weird losers

[deleted]

Re: Rust in the kernel is no longer experimental

#788

Post by maccard »

johncolanduoni wrote: post #758
maccard wrote: post #483

Earlier quoted context omitted.

How many of them are running Linux, and how many of them are running a modern kernel?

Almost none of them, but that doesn’t make them “places no one uses”, considering everyone who can see these words is looking at a device with a bunch of C firmware running on microcontrollers all over the place.

> but that doesn’t make them “places no one uses

You're putting words in my mouth there - that's not my point.

My point is that if they're not running linux, and the ones that are aren't running a modern kernel, then we shouldn't hold back development of modern platforms for ones that refuse to keep up.

Re: Rust in the kernel is no longer experimental

#789

Post by Xylakant »

menaerus wrote: post #778
littlestymaar wrote: post #711

Earlier quoted context omitted.

> especially in such a complex codebase You accidentally put the finger on the key point, emphasis mine. When you have a memory-unsafe language, the complexity of the whole codebase impact your ability to uphold memory-related invariants. But unsafe block are, by definition, limited in scope and assuming you design your codebase properly, they shouldn't interact with other unsafe blocks in a different module. So the…

No, this is just an example of confirmation bias. You're given a totally unrealistic figure of 1 vuln per 200K/5M LoC and now you're hypothesizing why that could be so. Google, for anyone unbiased, lost the credibility when they put this figure into the report. I wonder what was their incentive for doing so. > But unsafe block are, by definition, limited in scope and assuming you design your codebase properly, they s…

Let's take a simple example to illustrate how unsafe {} cuts down the review effort for many operations. Take a static mutable global variable (a global counter for example). Reading a static is safe, mutating it (increasing the counter) is not - it requires an unsafe {} block.

If you need to check which places mutate this global static you only need to check the unsafe parts of the code - you know that no other part of your code could mutate it, the compiler won't let you. If you have a bug that is related to mutating this static, then it might manifest anywhere in your code. But you know for certain that the root cause must be in one of your unsafe blocks - even if you don't know which one.

Good programming practice will cut down that effort even more by dictating that unsafe access should be grouped in modules. For example when binding to a C module (unsafe) you'd usally generate an unsafe wrapper with bindgen and then write a safe wrapper on top of that. Any access that tries to go around the safe wrapper would be frowned upon and likely fail review.

And again, the compiler will help you there: Any access that tries to bypass the safe api would need to be unsafe {} again and automatically receive extra scrutiny in a review, making it less likely to slip through.

Compare that to a C codebase where anything goes. A static might be mutated anywhere in your codebase, even through a pointer to it - meaning you can't even reliably grep for it. It may slip through review unnoticed because no attention is drawn to it and cause bugs that are hard to trace and reason about.

If you're writing embedded code, similar considerations apply - access to registers etc. require unsafe {}. But because access is unsafe {}, it's usually gated behind a safe api that is the boundary of the low-level code and the higher buisness logic. Unsurprisingly, these are critical parts of the code - hence they receive extra scrutiny and in our project, we allocate substantial review capacity on those. And the compiler will enforce that no safe code can circumvent the access layer.

The number you're tagging as unrealistic figure is the result of dedicated and careful design of language and compiler features to achieve exactly this outcome. It's not a random fluke, very clever people did sit down and thought about how to achieve this.

Re: Rust in the kernel is no longer experimental

#790

Post by sunshowers »

menaerus wrote: post #778
littlestymaar wrote: post #711

Earlier quoted context omitted.

> especially in such a complex codebase You accidentally put the finger on the key point, emphasis mine. When you have a memory-unsafe language, the complexity of the whole codebase impact your ability to uphold memory-related invariants. But unsafe block are, by definition, limited in scope and assuming you design your codebase properly, they shouldn't interact with other unsafe blocks in a different module. So the…

No, this is just an example of confirmation bias. You're given a totally unrealistic figure of 1 vuln per 200K/5M LoC and now you're hypothesizing why that could be so. Google, for anyone unbiased, lost the credibility when they put this figure into the report. I wonder what was their incentive for doing so. > But unsafe block are, by definition, limited in scope and assuming you design your codebase properly, they s…

You sound like my teacher in school, after my friend submitted an assignment so good my teacher thought he was cheating.
Post reply on HN