Live data from Hacker News

Supporting Linux kernel development in Rust

lwn.net

191–200 of 365 posts

Re: Supporting Linux kernel development in Rust

#191

I'm concerned about the gradual move from GCC to LLVM. The lack of copyleft protections on LLVM means that it's much more dependent on corporate sponsorship, and means that there's a risk that major improvements to LLVM compilers will only become available as proprietary products. People underestimate the role of copyleft licenses in preserving long-running FOSS products like GCC, Linux, etc.

> I'm concerned about the gradual move from GCC to LLVM. The lack of copyleft protections on LLVM means that it's much more dependent on corporate sponsorship, and means that there's a risk that major improvements to LLVM compilers will only become available as proprietary products. As someone who works within LLVM professionally, I don't think this is particularly likely -- compilers are massive and complicated beas…

> they're sufficiently general and easiest to maintain by releasing back upstream (where Apple, Google, and Microsoft will pay a small army of compiler engineers to keep them working).

...keep them working for their use-cases.

Re: Supporting Linux kernel development in Rust

#192

I'm concerned about the gradual move from GCC to LLVM. The lack of copyleft protections on LLVM means that it's much more dependent on corporate sponsorship, and means that there's a risk that major improvements to LLVM compilers will only become available as proprietary products. People underestimate the role of copyleft licenses in preserving long-running FOSS products like GCC, Linux, etc.

That makes me sad as well, but I also think it was largely self-inflicted. GCC took much too long to realize that people needed a common code-generation framework, and be willing to support that and work towards it. LLVM was designed from the start to be such a framework.

Re: Supporting Linux kernel development in Rust

#193
post #180

Earlier quoted context omitted.

> Vecdeque is implemented once in the standard library it is, what about other similar cases of using pointers in other data-structures and algorithms that are not part of the Rust standard library?

They are implemented once in a crate on crates.io and similarly re-used. Or in a code base like Linux with some unique requirements they'll be implemented once there and re-used many times there. As it turns out most code isn't datastructures with unique memory requirements (i.e. that can't be implemented in terms of other datastructures), most code just re-uses existing datastructures.

> they are implemented once in a crate on crates.io and similarly re-used.

this doesn't automatically prevent the code from containing CVE, does it?

> As it turns out most code isn't datastructures with unique memory requirements

most code that has been written in Rust so far. But when you enter a territory of OS kernels, unique memory requirements and data structures such as ring buffers and dynamic mutable trees are the things that have to be implemented according to the requirements, and we want them to be implemented as efficiently as C is capable of, and correctly. Falling back to "unsafe" is an option, but it's hardly an argument that supports the effort of bringing safety to such a kernel.

But let's say that we've solved pointer issues. What about dependent types and totality checking? ATS has that [1] [2]

[1] http://ats-lang.github.io/DOCUMENT/INT2PROGINATS/HTML/INT2PR...

[2] http://ats-lang.github.io/DOCUMENT/INT2PROGINATS/HTML/INT2PR...

Re: Supporting Linux kernel development in Rust

#194

Earlier quoted context omitted.

I am even more concerned about LLVM monoculture. LLVM is just a proprietary-able version of GCC. And thus compiler authors mostly focus on benchmarks while not caring about other metrics such as compilation speed, correctness and codebase quality / ease of contribution. There appears to be some tribal knowledge requirement to add a new target. With all the hype and apple funding LLVM gets, it could have been better.

I wouldn't say GCC has an edge on "codebase quality / ease of contribution". Doesn't RMS actively pushes against such metrics? https://gcc.gnu.org/legacy-ml/gcc/2014-01/msg00247.html

Having contributed to both (nothing major but not trivial one-liners either), my feeling is GCC definitely has a higher quality codebase and a more rigorous review process. While the latter doesn't always translate to the ease of contribution, I found that if you are persistent enough, your patches to GCC will be reviewed. On the other hand, I had my patches to LLVM ignored forever. Just my 2c based on experience.

Re: Supporting Linux kernel development in Rust

#195
post #160

Earlier quoted context omitted.

> wrapping/boxing/unboxing, as they have exactly the same native data representation Rust's Box and Option > have the same representation as a C pointer to the same type, so this is true of Rust too. You do have to wrap the type in source code to indicate what the ownership guarantees/contracts are. (Is this not also true of ATS? That is, if ATS code calls a C function that returns a pointer, how do you know how long…

> No, adding unsafe Rust does not undermine the safety argument But it does, see my answer in the thread below regarding ring buffers - https://news.ycombinator.com/item?id=24337184 > Is this not also true of ATS? That is, if ATS code calls a C function that returns a pointer, how do you know how long that pointer is valid for and whether you're expected to free it? the C implementation needs to be known, then the si…

> ring buffers in Rust use "unsafe" and at some point there was a logical CVE in vec_deque that broke one invariant of the unsafe block

First, you're talking about two different things here. Originally you were talking about incrementally adding Rust bindings to C code and how that obligated you to write "unsafe". Now you are talking about pure-Rust implementations of data structures that use "unsafe" for optimization purposes. The second one is much more dangerous.

Second, it does not undermine the safety argument - quite the opposite, as demonstrated by the fact that code that used VecDeque remained unchanged after the bug fix. Everything that the Rust compiler proved about code that used VecDeque, on the assumption that VecDeque was sound, remained just as proven after the bug was fixed and VecDeque was changed to be actually sound. There was no ex-falso-quodlibet problem. There was a clear delineation in the safety argument: on one side, a human was obligated to check that VecDeque was implemented soundly, and on the other side, the Rust compiler checked that every user of VecDeque did so soundly. The first part was done incorrectly and then fixed; the second part remained done correctly.

> the proof itself can indicate whether a pointer is stored/taken care of inside the C function or it should be taken care of outside the function (the latter is indicated by a prefixed "!" to the pointer type).

Then this is an axiom as input to the proof, not a proof itself. That is, ATS is no more capable than Rust of magically determining what the unstated invariants of C code are; it can only prove that certain things logically follow from certain assumptions. This is precisely what Rust does, too, except it is clearer than ATS because any use of such unchecked assumptions are clearly marked with the "unsafe" keyword.

In the linked ATS code, I see no indication, as a human reviewer, about which parts I should carefully audit (e.g., the annotation of the "extern fun") and which parts ATS has checked for me (e.g., the implementation of "string_to_base64"). In Rust, you can very quickly search a codebase for "unsafe" and see what needs auditing. (And in fact this technique empirically works well for finding unsoundness in production Rust code, and I can point to multiple examples of it. I wonder what people do when reviewing production ATS code.)

Again, I'm not saying ATS isn't cool. I'm not saying it wouldn't be great to have ATS support in the Linux kernel. Please work on this. There are, in fact, lots of things that ATS can do that Rust cannot do, and they are helpful to have. But I think the specific things you are claiming are things that Rust can do just fine or that ATS in fact cannot do.

Re: Supporting Linux kernel development in Rust

#196
post #191

Earlier quoted context omitted.

> I'm concerned about the gradual move from GCC to LLVM. The lack of copyleft protections on LLVM means that it's much more dependent on corporate sponsorship, and means that there's a risk that major improvements to LLVM compilers will only become available as proprietary products. As someone who works within LLVM professionally, I don't think this is particularly likely -- compilers are massive and complicated beas…

> they're sufficiently general and easiest to maintain by releasing back upstream (where Apple, Google, and Microsoft will pay a small army of compiler engineers to keep them working). ...keep them working for their use-cases .

> keep them working for their use-cases

Are you under the impression that C and C++ code within Apple, Microsoft, and Google is fundamentally different from C and C++ code elsewhere? Because it isn’t. Google’s engineers maintain LLVM’s ASan for their purposes, but those purposes happen to be everybody else’s as well.

Re: Supporting Linux kernel development in Rust

#197
post #127

Earlier quoted context omitted.

Please don't be repetitive in HN comments, though. Repetition is the enemy of curiosity and curiosity is the core value here. https://news.ycombinator.com/newsguidelines.html

What would be the proper way of informing those who are interested in the topic about available alternatives? I assume not everyone follows the same topics and there may be people who see the presented information and the relevant references for the first time.

At the very least, please include links to your previous comments, so that any refutations of your previous comments can be seen by others - otherwise we will all have to repeat ourselves until there is no room for any new discussion. (One of the other replies pointed out that you had not incorporated an objection that someone made to your previous comment.)

Re: Supporting Linux kernel development in Rust

#198
post #180

Earlier quoted context omitted.

They are implemented once in a crate on crates.io and similarly re-used. Or in a code base like Linux with some unique requirements they'll be implemented once there and re-used many times there. As it turns out most code isn't datastructures with unique memory requirements (i.e. that can't be implemented in terms of other datastructures), most code just re-uses existing datastructures.

> they are implemented once in a crate on crates.io and similarly re-used. this doesn't automatically prevent the code from containing CVE, does it? > As it turns out most code isn't datastructures with unique memory requirements most code that has been written in Rust so far. But when you enter a territory of OS kernels, unique memory requirements and data structures such as ring buffers and dynamic mutable trees ar…

> this doesn't automatically prevent the code from containing CVE, does it?

There are certainly tools to detect any known CVEs, and tools to help you audit the unsafe code that you are using. They don't provide rigorous proofs automatically of course, the same way the standard library is generally not rigorously proved to be correct.

> But when you enter a territory of OS kernels,

Nothing really changes, see redox as an example of a fairly "complete" kernel written in rust with a reasonably low amount of unsafe.

> we want them to be implemented as efficiently as C is capable of,

This is already the case with Rust's typical library system. Generic libraries in rust are not less efficient.

In the real world rust datastructures are typically more efficient in my experience because the clean separation makes it easy to optimize the hell out of them.

> What about dependent types

That's a tool not a problem - you have yet to provide an argument that they are necessary and real world experience says that Rust achieves a meaningful amount of safety and convenience without them.

> totality checking

That's also a tool not a problem, and you could argue that rust's ADTs (enums) give you a weak version of it.

Re: Supporting Linux kernel development in Rust

#199
post #190

Earlier quoted context omitted.

> We can equally well say that that "Rust formally verifies your safe functions", and that this reduces how much code review is required... it's a matter of degree and specifics. You cannot say that while this kind of CVE is possible https://gts3.org/2019/cve-2018-1000657.html and as long as Rust is not capable of performing safe pointer manipulations. The above issue may be solved for VecDeque in stdlib, but what ab…

"In the wild" one hardly ever has to write unsafe Rust data structures and algorithms. > If the motivation to bring a new tool is pronounced as "let's make it safe", why should we allow the argument to fallback into the "unsafe Rust" territory? Because going from 0% of code proven safe by the compiler to 99% is very valuable. Verifying the remaining code is desirable but not as valuable as what Rust already provides.…

> "In the wild" one hardly ever has to write unsafe Rust data structures and algorithms.

one has to do it all the time if cyclic mutable graphs or specific buffers/caches are involved.

> Verifying the remaining code is desirable but not as valuable as what Rust already provides.

How do we know that? What are the criteria and the thresholds that lead us to that conclusion? Is it true for all fields where the language can be used? What should we do about inability to express more precise constraints at compile time? Shall we stop on Rust, or try to embrace more powerful tools that already support Dependent Types in low-level systems programming? These checks enable a whole new world of expressive powers and correctness guarantees, even compared to the cool borrow-checker. ATS supports them today [1]

[1] http://ats-lang.sourceforge.net/DOCUMENT/INT2PROGINATS/HTML/...

Re: Supporting Linux kernel development in Rust

#200

Earlier quoted context omitted.

If I ommited that first part of the sentence the points made afterwards would remain the same. I did so to indicate that it's not the first time the Rust community suggests that something should be (re-)written in Rust for memory-safety reasons, in a project that has a well-established C-codebase. I'd argue it's the same "Just install Linux" situation that you mention, and that "Linux" may not be the answer.

The Rust community is not advocating that, it’s kernel developers that are interested in writing new components in Rust. So no proposal of rewriting by any Rust developers

And 'ghostwriter stated elsewhere in this thread that the previous round of these comments was in reply to an article talking about how QEMU should start to develop new device backends as out-of-process Rust programs - which also was not about outsiders proposing a complete rewrite, either.

https://news.ycombinator.com/item?id=24133292

http://blog.vmsplice.net/2020/08/why-qemu-should-move-from-c...

Perhaps the most long-lasting harm of the Rust Evangelism Strike Force has been the rise of the Anything But Rust Counterinsurgency, who sees that someone, somewhere, is thinking about Rust, and needs to convince them that they're making a mistake.

Post reply on HN