Live data from Hacker News

Kerla: Monolithic kernel in Rust, aiming for Linux ABI compatibility

github.com

171–180 of 196 posts

Re: Kerla: Monolithic kernel in Rust, aiming for Linux ABI compatibility

#171
post #169

Earlier quoted context omitted.

> Rust's memory model largely makes those isolated address spaces unnecessary, I think you're confusing several concepts. Rust's "memory model" is the C++11 memory model. Maybe you meant the borrow checker and the distinction between shared and exclusive references? Yes, those make it much harder to accidentally screw up other parts of your address space, but it's downright trivial to do it on purpose – you just need…

I think we have to assume that people aren't putting malicious code into the kernel, don't we?

The kernel will have a lot of "unsafe" (the keyword) code.

Re: Kerla: Monolithic kernel in Rust, aiming for Linux ABI compatibility

#172
post #89

Earlier quoted context omitted.

I don't see why we need to pretend that every hobby has the potential for greatness. Being a hobby is a good enough end to itself. In fact in this instance I think it's a little disingenuous to quote Linux and say it could happen again. The industry is totally different now. There's much more competition than there was when Linux was released and that competition is much more mature too. Plus the bar for a production…

> In fact in this instance I think it's a little disingenuous to quote Linux and say it could happen again. Seems apropos to me, given the fact that a Linux ABI compatible hobby project is under discussion - and that everyone here is familiar with the famous Usenet announcement. > The industry is totally different now. There's much more competition than there was when... I wonder how you define "competition"? Because…

> I wonder how you define "competition"? Because there were way more operating systems in use then, and the industry was far more fractured.

There is way more competition now:

  + Linux (countless distributions)
  + FreeBSD
  + OpenBSD
  + NetBSD
  + DragonflyBSD
  + HardenedBSD
  + Darwin
  + Minix 3 (which wasn't free when Linux was released)
  + Illumos
  + OpenIndiana
  + Nexenta OS
  + SmartOS
  + ...and many others based off OpenSolaris / Illumos
This isn't even an exhaustive list of UNIX-like platforms that are new since Linux and free.

Don't conflate standardisation of the industry with a lack of options. More options do exist today and are in use (eg some games consoles run FreeBSD, Netflix uses BSD, Nexenta is used in some enterprise storage solutions, Darwin may not be used in any free capacity but macOS is clearly used heavily by HN readers, and so on and so forth).

Moreover, I've used FreeBSD, Solaris, OpenSolaris, Nexenta and OpenBSD on production systems over the last 10 years (and the list gets more esoteric if we look past 10 years). So just because you might exist in a Linux-only ecosystem it doesn't mean that's the case for the entire IT industry.

Re: Kerla: Monolithic kernel in Rust, aiming for Linux ABI compatibility

#173
post #169
post #153

Earlier quoted context omitted.

Historically, one of the main arguments against microkernels is that having a kernel that's broken up into a lot of threads that each have their own hardware-enforced memory protection and use message passing to communicate with each other is dreadfully inefficient. All those context switches are expensive, and copying all that data around when you send messages is expensive. Rust's memory model largely makes those i…

> Rust's memory model largely makes those isolated address spaces unnecessary, I think you're confusing several concepts. Rust's "memory model" is the C++11 memory model. Maybe you meant the borrow checker and the distinction between shared and exclusive references? Yes, those make it much harder to accidentally screw up other parts of your address space, but it's downright trivial to do it on purpose – you just need…

The point was to use compile-time checks instead of a MMU to separate kernel services.

The concept is interesting (if maybe untested) it allows to have explicit opt-in sharing while blocking accidental sharing

Re: Kerla: Monolithic kernel in Rust, aiming for Linux ABI compatibility

#174
post #163
post #161

Earlier quoted context omitted.

> There's much more competition than there was when Linux was released and that competition is much more mature too. I think it's the opposite. When Linux was released, basically every major tech company had their own variant of Unix. Now almost everyone has migrated to Linux or Windows. Competing with modern Linux by creating a drop-in replacement for Linux would be a pretty difficult task given that people have bee…

Follow up to add: another way to compete with Linux is on security. The Linux kernel generally has a pretty good security record I think, but there have been plenty of serious bugs over the years. How many exploitable array-out-of-bounds errors or use-after-free errors remain in the Linux kernel? No one knows. If you can rule those out by using a safer language, that might be compelling to a lot of users who care abo…

If OpenBSD has taught us anything, it's that when you need to start hardening at that level, C stops becoming weakest link and actually the design of the broader UNIX ABIs are the bigger problem. This is why things like selinux and cgroups exist in Linux -- POSIX ABIs are about as secure as Win32 APIs in Windows and thus you need to take additional steps to isolate your running processes if you really care about them behaving.

Re: Kerla: Monolithic kernel in Rust, aiming for Linux ABI compatibility

#175
post #169

Earlier quoted context omitted.

> Rust's memory model largely makes those isolated address spaces unnecessary, I think you're confusing several concepts. Rust's "memory model" is the C++11 memory model. Maybe you meant the borrow checker and the distinction between shared and exclusive references? Yes, those make it much harder to accidentally screw up other parts of your address space, but it's downright trivial to do it on purpose – you just need…

I think we have to assume that people aren't putting malicious code into the kernel, don't we?

But that's one of the big promises of micro-kernels: You don't have to have 100% trust in the driver for that gamepad you bought from some unknown company.

What about that WiFi driver with 10,000s of lines of code? Are you absolutely sure there isn't some code in it that accidentally or purposefully leaks the rest of your kernel data structures to the outside world? With drivers in their own isolated address space, you wouldn't have to be.

Re: Kerla: Monolithic kernel in Rust, aiming for Linux ABI compatibility

#176
post #173
post #169

Earlier quoted context omitted.

> Rust's memory model largely makes those isolated address spaces unnecessary, I think you're confusing several concepts. Rust's "memory model" is the C++11 memory model. Maybe you meant the borrow checker and the distinction between shared and exclusive references? Yes, those make it much harder to accidentally screw up other parts of your address space, but it's downright trivial to do it on purpose – you just need…

The point was to use compile-time checks instead of a MMU to separate kernel services. The concept is interesting (if maybe untested) it allows to have explicit opt-in sharing while blocking accidental sharing

> The point was to use compile-time checks instead of a MMU to separate kernel services.

Rust's compile time checks cannot replace the MMU. There were experimental SAS (single address space) operating systems which didn't depend on the MMU for protection, but they used "managed" languages, i.e., all code was executed in a sandboxed JIT.

Re: Kerla: Monolithic kernel in Rust, aiming for Linux ABI compatibility

#177
post #169
post #153

Earlier quoted context omitted.

Historically, one of the main arguments against microkernels is that having a kernel that's broken up into a lot of threads that each have their own hardware-enforced memory protection and use message passing to communicate with each other is dreadfully inefficient. All those context switches are expensive, and copying all that data around when you send messages is expensive. Rust's memory model largely makes those i…

> Rust's memory model largely makes those isolated address spaces unnecessary, I think you're confusing several concepts. Rust's "memory model" is the C++11 memory model. Maybe you meant the borrow checker and the distinction between shared and exclusive references? Yes, those make it much harder to accidentally screw up other parts of your address space, but it's downright trivial to do it on purpose – you just need…

I meant the ownership model.

I'm also imagining a usage model where all the various components of the microkernel come from the same place and have passed various minimum standards like "doesn't have any unsafe blocks". If some program really needs an unsafe block to do something, then that goes into a (hopefully small) library of heavily-scrutinized functions that do dangerous things but expose safe interfaces.

If we compare with the Linux kernel: in Linux, any part of the kernel can clobber memory in any other part; there's no protection at all. Users generally accept that because the Linux kernel is a pretty well-run project, but still bugs slip in from time to time. If instead you imagine a system where the compiler simply doesn't allow one part of the kernel to clobber memory that belongs to some other part, that would be a significant improvement over what Linux offers.

Anyways, security mechanisms don't necessarily have to be enforced at run-time if they can be enforced at compile-time. That assumes you compile the code yourself with a trusted compiler, or you get the binaries from a trusted source. (Though unfortunately Spectre has become a big problem for compiler-enforced data isolation within one big shared address space; I'm not sure whether compilers these days can do code generation in such a way that it doesn't suffer from sidechannel attacks, or if for the foreseeable future it's just something that software developers have to be wary of.)

In theory you might eventually be able to run device drivers from random people or run arbitrary applications in the same address space as the kernel, but that would require a lot of faith in the compiler to reject anything that could be a security hole. It's an interesting model, though; a pure unikernel approach everything shares the same page tables and there's no need to swap them in or out when context switches happen. And invoking a system call can be as cheap as executing an ordinary function. It might even be inlined by the compiler.

Re: Kerla: Monolithic kernel in Rust, aiming for Linux ABI compatibility

#178
post #87

Earlier quoted context omitted.

> it would allow me to additionally log something If you don't have any memory your allocations are all failing. When you assemble the log message, the allocation needed to do that fails. Bang, double fault. Now, often people don't really mean they want allocations to be able to fail generally, they're just thinking about that code they wrote that reads an entire file into RAM. If it was a 100GB file that would be a…

People say that "well if allocations fail all bets are off" but can't you pre-allocate memory for error handling? Like sit down, figure out all the things you'll want to do on an allocation failure, and once you have determined that you slice a little chunk of memory when you start your app (and maybe _that_ fails and you can't do anything). and when you hit a failure you do your think, then tear stuff down.

It's what we used to do in the days when 4MB was a lot of memory. Batch programs would just abort but interactive programs had to have enough reserve to fail gracefully, possibly unwinding and releasing things until they could operate better.

Now that I see interactive programs taking a gigabyte and the system being ok, I guess we're in a different regime.

Re: Kerla: Monolithic kernel in Rust, aiming for Linux ABI compatibility

#179

Rewriting the Linux kernel in rust would be a rehash of why we ended up with Linux in the first place: rewriting an older OS from scratch. I would love to see a solid implementation of a real micro kernel based OS based on a message passing core in Rust to become a viable alternative to the now 50 year old Unix architecture. This would give us a possibility of some real progress on the security and process scheduling…

Mach's internals didn't follow the Unix model, and was by almost any metric a failure. There have been several other "real micro kernel based OS based on a message passing core", and they too, by almost any metric, have been a failure. Process scheduling in Linux in 2021 is far, far ahead of anything in any other OS, ukernel or MP-based. The idea that there hasn't been progress in this area is really completely absur…

Mach is of course doing quite well and moving back towards being microkernel-y as hardware gets fast enough to support it and Apple needs a better security story.

Re: Kerla: Monolithic kernel in Rust, aiming for Linux ABI compatibility

#180
post #175

Earlier quoted context omitted.

I think we have to assume that people aren't putting malicious code into the kernel, don't we?

But that's one of the big promises of micro-kernels: You don't have to have 100% trust in the driver for that gamepad you bought from some unknown company. What about that WiFi driver with 10,000s of lines of code? Are you absolutely sure there isn't some code in it that accidentally or purposefully leaks the rest of your kernel data structures to the outside world? With drivers in their own isolated address space, y…

That was the point behind Singularity OS[1] IIRC. Using managed code which could be verified safe by the OS using static analysis.

Can't find it right now, but I recall reading they implemented a network driver with minimal overhead compared to a traditional OS. While they used message passing, a lot of overhead could be eliminated due to the assumptions of safe code.

[1]: https://en.wikipedia.org/wiki/Singularity_%28operating_syste...

Post reply on HN