Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Writing an OS in Rust: Async/Await

os.phil-opp.com

Post reply on HN
Print view View original

11–20 of 107 posts

Re: Writing an OS in Rust: Async/Await

#11

Post by zackmorris »

I've shied away from async/await because I haven't seen a good writeup on how to make it deterministic. Come to think of it, all of the times I've encountered it, there was no way to analyze the codepaths and prove that every exceptional situation and failure mode was handled. Maybe I missed something along the way?

So my feeling about it is that it may turn out to be an evolutionary dead end, or anti-pattern at the least. My gut feeling is that async/await is functionally equivalent to the the synchronous/blocking/coroutine/channel system of other languages like Go. Could we write a thin wrapper that converts async/await to that or vice versa?

This is the primary reason why I've stuck to synchronous/blocking PHP with all of its flaws instead of Node.js. I think this is a fundamental thing that shouldn't be glossed over and accepted so readily into other languages.

Re: Writing an OS in Rust: Async/Await

#12

Post by steveklabnik »

zackmorris wrote: post #11

I've shied away from async/await because I haven't seen a good writeup on how to make it deterministic. Come to think of it, all of the times I've encountered it, there was no way to analyze the codepaths and prove that every exceptional situation and failure mode was handled. Maybe I missed something along the way? So my feeling about it is that it may turn out to be an evolutionary dead end, or anti-pattern at the…

What do you mean by "deterministic" here?

> Come to think of it, all of the times I've encountered it, there was no way to analyze the codepaths and prove that every exceptional situation and failure mode was handled.

There is nothing special about async/await with regards to this in Rust, at least if I'm understanding you correctly. Async functions can return Results like any other function for recoverable errors, and can panic like any other function for un-recoverable errors.

> My gut feeling is that async/await is functionally equivalent to the the synchronous/blocking/coroutine/channel system of other languages like Go.

It depends on what level of abstraction you're talking about. For Rust, which cares a lot about details, they're not. I gave a talk comparing and contrasting all this stuff here: https://www.infoq.com/presentations/rust-2019/

Re: Writing an OS in Rust: Async/Await

#13

Post by bestouff »

hope-striker wrote: post #10
amelius wrote: post #4

An OS with async/await sounds awfully similar to Windows 3.1 with its cooperative multitasking model. What's old is new again ...

Didn't most old operating systems use cooperative multitasking? I remember, at least, that classic Mac OS (i.e. pre-OSX) didn't use preemptive multitasking, either. Anyway, this SO answer[0] explains why early Linux, much like the hobby kernel in this article, used cooperative scheduling inside the kernel, and only preempted user-space stuff. [0]: https://stackoverflow.com/a/16766562

MacOS may have been a little late. Windows NT, OS/2, Linux did preemptive multitasking since beginning of 90s, even AmigaOS had it in the 80s.

Re: Writing an OS in Rust: Async/Await

#14

Post by AnIdiotOnTheNet »

bluejekyll wrote: post #9
AnIdiotOnTheNet wrote: post #7

Earlier quoted context omitted.

Well, I'm not well versed in the details, but I do wonder if cooperative multitasking isn't a better idea nowadays in the age of ubiquitous multicore.

As Steve’s sibling comment points out, this is only for the kernel. The article mentions preemptive threading for the user space as still being desirable.

I'm not convinced that's actually true though, is what I mean. The big drawback of cooperative is just that a single process can monopolize resources, which in the singlecore days mean the system became unresponsive and you had no way to recover from it. That isn't really true now with multicore. As long as the OS has a core to work on it can terminate or otherwise deal with misbehaving processes.

Again, this is based on my admittedly limited understanding, but I haven't yet seen good reasoning why pre-emptive is still preferred.

Re: Writing an OS in Rust: Async/Await

#15

Post by jerf »

Is async/await a good idea for an OS kernel, even a toy one? Cooperative multitasking tends to break down at scale, because the probability that all the "threads" you're cooperating with are playing nice goes to zero as the number of threads increases. An OS will tend to have a concentrated number of the pathological cases in it as it deals with hardware and all the other hardest timing & concurrency problems.

It's a viable option for user-space programs because you can far more tightly characterize them and program them from top to bottom to work with that paradigm nice. Embedding islands of cooperative multitasking in a sea of pre-emptive multitasking seems to make a lot more sense than the other way around.

However, this post is a question, not a statement. If, for example, a Linux kernel developer posts "nah, it's no biggie, the Linux kernel is effectively structured the same", for instance, I would not quibble.

Re: Writing an OS in Rust: Async/Await

#16

Post by jklehm »

steveklabnik wrote: post #5
amelius wrote: post #4

An OS with async/await sounds awfully similar to Windows 3.1 with its cooperative multitasking model. What's old is new again ...

Well, there's a big difference: > Using async/wait, we now have basic support for cooperative multitasking in our kernel. While cooperative multitasking is very efficient, it leads to latency problems when individual tasks keep running for too long and thus prevent other tasks to run. For this reason, it makes sense to also add support for preemptive multitasking to our kernel. > In the next post, we will introduce t…

Isn't that driving against the lesson learned from the past though? 3rd party drivers can not be trusted to behave and hang the kernel when doing so with cooperative multitasking.

Re: Writing an OS in Rust: Async/Await

#17

Post by steveklabnik »

AnIdiotOnTheNet wrote: post #14
bluejekyll wrote: post #9

Earlier quoted context omitted.

As Steve’s sibling comment points out, this is only for the kernel. The article mentions preemptive threading for the user space as still being desirable.

I'm not convinced that's actually true though, is what I mean. The big drawback of cooperative is just that a single process can monopolize resources, which in the singlecore days mean the system became unresponsive and you had no way to recover from it. That isn't really true now with multicore. As long as the OS has a core to work on it can terminate or otherwise deal with misbehaving processes. Again, this is base…

My understanding is that you've now moved the monopolized case from 1 to N, where N is the number of cores. N is usually pretty small.

The laptop I'm writing this on has four cores, and the Chrome instance has 17 threads running...

Re: Writing an OS in Rust: Async/Await

#18

Post by eximius »

zackmorris wrote: post #11

I've shied away from async/await because I haven't seen a good writeup on how to make it deterministic. Come to think of it, all of the times I've encountered it, there was no way to analyze the codepaths and prove that every exceptional situation and failure mode was handled. Maybe I missed something along the way? So my feeling about it is that it may turn out to be an evolutionary dead end, or anti-pattern at the…

What do you mean by functionally equivalent?

Channels are just queues and passing methods. Rust style futures are closer to continuations. Both are powerful and can accomplish neat taska, but not ABI compatible or even used the same way.

Re: Writing an OS in Rust: Async/Await

#19

Post by steveklabnik »

jklehm wrote: post #16
steveklabnik wrote: post #5

Earlier quoted context omitted.

Well, there's a big difference: > Using async/wait, we now have basic support for cooperative multitasking in our kernel. While cooperative multitasking is very efficient, it leads to latency problems when individual tasks keep running for too long and thus prevent other tasks to run. For this reason, it makes sense to also add support for preemptive multitasking to our kernel. > In the next post, we will introduce t…

Isn't that driving against the lesson learned from the past though? 3rd party drivers can not be trusted to behave and hang the kernel when doing so with cooperative multitasking.

I am not sure what Phil's overall kernel design is; you're assuming a monolithic kernel, but not every kernel has drivers in kernel space.

Re: Writing an OS in Rust: Async/Await

#20

Post by jklehm »

steveklabnik wrote: post #19
jklehm wrote: post #16

Earlier quoted context omitted.

Isn't that driving against the lesson learned from the past though? 3rd party drivers can not be trusted to behave and hang the kernel when doing so with cooperative multitasking.

I am not sure what Phil's overall kernel design is; you're assuming a monolithic kernel, but not every kernel has drivers in kernel space.

Fair point, looking forward to following along :)
Post reply on HN