Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Supporting Linux kernel development in Rust

lwn.net

Post reply on HN
Print view View original

111–120 of 365 posts

Re: Supporting Linux kernel development in Rust

#111

Post by Ericson2314 »

ghostwriter wrote: post #30

I won't get tired of repeating the same comment in every topic that suggests Rust to be a great replacement of C in existing projects , that it isn't. The safety guarantees that Rust provides are neither unique nor complete, and if we discuss the amount of effort necessary for bringing new interfaces like the one this article mentions ("one can define a kmalloc_for_rust() symbol containing an un-inlined version"), we…

To me the benefit of Rust is less safety, than better abstractions and modularity. Linux has been growing in complexity without much of a breaking point, and the discipline and separation of concerns high level languages could help with is a huge long-term benefit.

(For example, say we could one day make a bunch of basic Linux drivers somewhat kernel agnostic? That would be amazing.)

Now, ATS is I hear a fine language, but I think the ability to write good, and especially abstract, ATS is sadly too far outside of most kernel dev's skill set. Even among PL researchers, there is a tendency to write monolithic Coq---to wit, is there a Hackage or crates.io for Coq?---because abstraction are hard and the academic paper economy doesn't really reward it.

Rust is just easy enough, and with enough crates.io momentum, that I hope the relative cost of reusable vs unreusable code will be less.

And finally, it takes a lot of political will to introduce a new language in any existing project, let alone one as big and storied as Linux. Like it or not, but general popularity of the language absolutely does help with that.

Re: Supporting Linux kernel development in Rust

#112

Post by bsder »

ghostwriter wrote: post #81
kbenson wrote: post #61

Earlier quoted context omitted.

> I won't get tired of repeating the same comment in every topic that suggests Rust to be a great replacement of C in existing projects, that it isn't. That's not what this is about, so even though you aren't tired of posting this, I'm not sure it's warranted. This is about the kernel supporting other languages in select places where it makes sense, such as kernel modules, where there's already an API in place to con…

> It's also not limited to Rust, it's about providing support for multiple other languages that could be beneficial in those locations But what are these benefits that other languages bring on the table? From the article it appears that proponents of the new toolchain mention memory-safety as the primary concern: > They focused on security concerns, citing work showing that around two-thirds of the kernel vulnerabili…

> So the natural follow-up question to that concern, it seems to me, would be whether there's something in the existing C/GCC toolchain (or around it) that can resolve the raised concerns without adopting a whole new toolchain (Rust/LLVM)

The answer to this question appears to be: no.

Memory errors sit somewhere around 2/3 to 3/4 of errors irrespective of codebase when C is involved (this has been consistent in things like the Linux kernel as well as the Windows codebase). C++ hasn't seemed to have changed that, either.

Remember, Rust IS Mozilla's answer to this question.

> without having to extend the existing interface with language-specific suffixes like "kmalloc_for_rust".

The requirement to do that is pointing out the that kernel API is poor and is too intimately tied to the build chain. We have seen this before--when Linux got ported to Alpha, for example.

The point is to fix the API so that more languages than just Rust can be used.

I suspect that more "system programming" languages may appear once Rust does the difficult groundwork of breaking this kind of intimate toolchain dependence. As things currently stand, why develop a system programming language given that no one will ever use it?

Re: Supporting Linux kernel development in Rust

#113

Post by darksaints »

ghostwriter wrote: post #30

I won't get tired of repeating the same comment in every topic that suggests Rust to be a great replacement of C in existing projects , that it isn't. The safety guarantees that Rust provides are neither unique nor complete, and if we discuss the amount of effort necessary for bringing new interfaces like the one this article mentions ("one can define a kmalloc_for_rust() symbol containing an un-inlined version"), we…

Reading through your various replies to comments, it seems your argument boils down to the FFI issue, and I find that a little bizarre. Sure, ATS can inline library calls due to the lack of an FFI, and maybe it has a cleaner interface to those libraries, but now you've introduced a completely different set of problems: now you have to tell the compiler how to prove your code.

In my experience, learning the fundamental concepts behind theorem proving is a lot harder of a problem than writing an FFI interface to a library. In fact, that problem is so easily solved that there are tools that do 99% of the work for you.

And sure, maybe rust will never be able to inline C code, but inlining function calls is such a trivially small performance gap that most applications would never know the difference.

I'm not convinced that there are better alternatives for contributing to or incrementally rewriting an existing code base. FFI is for most purposes a trivial impediment compared to the alternatives.

Re: Supporting Linux kernel development in Rust

#114

Post by geofft »

kazinator wrote: post #102
kbenson wrote: post #61

Earlier quoted context omitted.

> I won't get tired of repeating the same comment in every topic that suggests Rust to be a great replacement of C in existing projects, that it isn't. That's not what this is about, so even though you aren't tired of posting this, I'm not sure it's warranted. This is about the kernel supporting other languages in select places where it makes sense, such as kernel modules, where there's already an API in place to con…

> They are simply making sure those interfaces are not actively hostile to languages that aren't C ... I.e. not actively small and fast, like an OS should be.

1. This claim needs benchmarking.

2. Linux is already moving in the direction of having easier-to-use internal interface over small, fast, and tricky ones. For kmalloc in particular, the memory management docs https://www.kernel.org/doc/html/latest/core-api/memory-alloc... recommends kzalloc (which zeroes data) and suggests a function called kvmalloc which calls either kmalloc or vmalloc depending on the size of the allocation. It unconditionally recommends kvfree (which handles alloations from either kmalloc or vmalloc) instead of asking you to use kfree or vfree as appropriate.

3. In this particular case, kmalloc is inlined because it calls krealloc(NULL, ...). This could almost certainly be equivalently handled by inlining the other side of it. (Again, benchmarking.)

4. LTO is a thing.

Re: Supporting Linux kernel development in Rust

#115

Post by stmw »

rumanator wrote: post #63
stmw wrote: post #48

Earlier quoted context omitted.

ATS certainly looks interestinb, but it's an academic language (of which there are many) that most people probably haven't heard of... At some point, the momentum of a new programming language is just as important -- in practical terms -- as its formal attributes and qualities.

Your centering your criticism of ATS on popularity. Does an investment on PR trumps technical merit?

Popularity is not quite the same as PR. But yes, it's about the number of engineers and projects using it, and the growth of such adoption. If technical merits were the only criteria for making a difference, I think it's fair to say we would have a very different technology ecosystem today. :)

Re: Supporting Linux kernel development in Rust

#116

Post by geofft »

ogoffart wrote: post #91

> The ubiquitous kmalloc() function, for instance, is defined as __always_inline, meaning that it is inlined into all of its callers and no kmalloc() symbol exists in the kernel symbol table for Rust to link against. This problem can be easily worked around — one can define a kmalloc_for_rust() symbol containing an un-inlined version but performing these workarounds by hand would result in a large amount of manual wo…

Yup - that got a very brief mention in the article as "Google is working on automated C++ conversions"; there's also https://github.com/google/autocxx which builds on top of it.

C++ is a richer language than C and has more information about ownership (e.g., if you return a std::string you know how ownership works; if you have two arguments that are a char * and a length it's much less clear), but additional annotations in the kernel's C headers to convey this level of information in a machine-parseable way would be awesome.

Re: Supporting Linux kernel development in Rust

#117

Post by kbenson »

ghostwriter wrote: post #81
kbenson wrote: post #61

Earlier quoted context omitted.

> I won't get tired of repeating the same comment in every topic that suggests Rust to be a great replacement of C in existing projects, that it isn't. That's not what this is about, so even though you aren't tired of posting this, I'm not sure it's warranted. This is about the kernel supporting other languages in select places where it makes sense, such as kernel modules, where there's already an API in place to con…

> It's also not limited to Rust, it's about providing support for multiple other languages that could be beneficial in those locations But what are these benefits that other languages bring on the table? From the article it appears that proponents of the new toolchain mention memory-safety as the primary concern: > They focused on security concerns, citing work showing that around two-thirds of the kernel vulnerabili…

> But what are these benefits that other languages bring on the table? From the article it appears that proponents of the new toolchain mention memory-safety as the primary concern:

Of the venn diagram of languages that provide more safety than C (and that is, either require it or make it hard to bypass), and languages that people know know or are likely to know in the semi-near future, Rust and Go are probably the big contenders.

ATS may be easier to interface with the kernel in, but is random peripheral company writing a kernel module driver likely to know it and use it? Theoretically, everyone can write most or all of their modules in Coq, right? So why don't they? Why is ATS different?

> So the natural follow-up question to that concern, it seems to me, would be whether there's something in the existing C/GCC toolchain (or around it) that can resolve the raised concerns without adopting a whole new toolchain (Rust/LLVM)

You're mistaking the aims of a project to specifically adapt Rust to the linux kernel and what they focused on. This note is frm a Rust initiative, not a kernel initiative, why would they focus on doing something in a different language?

> and without having to extend the existing interface with language-specific suffixes like "kmalloc_for_rust".

I addressed this in the last comment. Did you have something new to bring to this aspect of the discussion? It feels like you're ignoring how I noted that this is but one option people are discussing, and using it as a reason why the whole thing is a bad idea. That doesn't make sense.

Re: Supporting Linux kernel development in Rust

#118

Post by [deleted] »

geofft wrote: post #114
kazinator wrote: post #102

Earlier quoted context omitted.

> They are simply making sure those interfaces are not actively hostile to languages that aren't C ... I.e. not actively small and fast, like an OS should be.

1. This claim needs benchmarking. 2. Linux is already moving in the direction of having easier-to-use internal interface over small, fast, and tricky ones. For kmalloc in particular, the memory management docs https://www.kernel.org/doc/html/latest/core-api/memory-alloc... recommends kzalloc (which zeroes data) and suggests a function called kvmalloc which calls either kmalloc or vmalloc depending on the size of the…

[deleted]

Re: Supporting Linux kernel development in Rust

#119

Post by ghostwriter »

johncolanduoni wrote: post #83
ghostwriter wrote: post #68

Earlier quoted context omitted.

It's not only inclusion of the headers. ATS compiles to C, and it can inline C without FFI. The benefit that you receive is the ability to introduce gradual automated proofs around unchanged stable interfaces that are known to be safe. You start with inlining everything but the small core that is proven to be safe. Then you can iterate indefinitely at the desired pace to bring new layers of formally verified API in p…

This may need fewer code changes (if the code in question is correct and the interfaces can possible be implemented in a safe way, that is) but it still requires developers to write proofs in a dependent type theory. I’ve only looked at ATS briefly, but I’m familiar with a lot of the other languages in the space (Idris, Fstar, etc.). Are ATS’s proof tactics drastically more productive/brief than those (F* in particul…

My experience may be too specific to reason about productiveness with ATS when it comes to other developers. I knew some Haskell and I've already read the Idris book when I discovered ATS, and I knew some C from university courses. The proofs require the same mental model, totality check principles are the same, resource utilisation principles are the same, so for me the difference when comparing these two langauges is mostly about the things available apart from the common features: ATS prelude has many predefined constraint functions that facilitate low-level programming with pointers over unboxed data.

I haven't tried F*, but from quick googling of "aliasing", it seems that a similar kind of checks can be achieved with view-changes in ATS (but please correct me if I'm missing the point) - http://ats-lang.sourceforge.net/DOCUMENT/INT2PROGINATS/HTML/...

Re: Supporting Linux kernel development in Rust

#120

Post by ghostwriter »

fluffy87 wrote: post #109
johncolanduoni wrote: post #83

Earlier quoted context omitted.

This may need fewer code changes (if the code in question is correct and the interfaces can possible be implemented in a safe way, that is) but it still requires developers to write proofs in a dependent type theory. I’ve only looked at ATS briefly, but I’m familiar with a lot of the other languages in the space (Idris, Fstar, etc.). Are ATS’s proof tactics drastically more productive/brief than those (F* in particul…

I’ve used ATS, Idris and Rust, and I find ATS way more cumbersome to use for writing proofs than idris. I’ve never used ATS to do what the GP is suggesting of incrementally wrapping C with proofs, but I can’t imagine this being simpler than just rewriting the C in Rust. You wouldn’t get the same proofs, but you would get memory and thread safety, which for many apps would be an incremental improvement. I haven’t taug…

> but I can’t imagine this being simpler than just rewriting the C in Rust.

sometimes it's impossible without falling back to unsafe Rust, which kind of undermines the initial incentive. C codebases heavily utilise pointer arithmetic programming.

Simplicity is a good trait though, and there are a few promising initiatives in that regard in ATS3 - https://github.com/githwxi/ATS-Xanadu#project-description

Post reply on HN