Live data from Hacker News

Writing an OS in Rust: Async/Await

os.phil-opp.com

101–107 of 107 posts

Re: Writing an OS in Rust: Async/Await

#101
post #71

Earlier quoted context omitted.

A state machine's control flow is even easier to follow, although more verbose.

I disagree, I find state machines almost impossible to reason about. Has it already been through state X? Who knows. Will it come back to this state in the future? Maybe. Is there a route from state Y to state Z? Shrug. State machines are effectively goto writ large, and there's a reason we switched to structured programming.

I wish I could upvote you 100. State machines are fine up to some number of states, on the order of 10. Beyond that, they become incredibly difficult to reason about in real programs because each state may have side effects with program state.

What ends up happening is that exceptional situations happen and the state machine becomes muddled with special cases. So for example, say you're writing a networked game of Tetris or Pacman, something with a relatively small state machine. You get all done but realize that you forgot to handle people joining or leaving the game, or the game closing or jumping ahead when a remote player beats the level and the game needs to go back to the title screen. The state machine starts getting all of these checks interspersed with the game logic.

Typically this doesn't happen with sync/blocking code though. You write an ordinary main loop, adding checks for these exceptional situations just like any other event you watch for. In my experience, the sync/blocking code is roughly an order of magnitude smaller and easier to reason about than a state machine. But more importantly, it can be scaled infinitely, because you can always add tracing or step through it in a debugger as you normally would. But a large state machine is just exactly what you described: a large switch command of gotos. You can't just look at it and know where the code came from or what conditions got it there. You have to derive that history by hand in your head.

And since async/await is more conceptually equivalent to a state machine than a coroutine or sync/blocking code, that severely limits how well we can reason about large promise chains. See my comment to jcranmer on this thread for a bit more insight about the internals of this:

https://news.ycombinator.com/item?id=22731043

Re: Writing an OS in Rust: Async/Await

#102

Earlier quoted context omitted.

What do you mean by "deterministic" here? > Come to think of it, all of the times I've encountered it, there was no way to analyze the codepaths and prove that every exceptional situation and failure mode was handled. There is nothing special about async/await with regards to this in Rust, at least if I'm understanding you correctly. Async functions can return Results like any other function for recoverable errors, a…

Well, I'm coming from the Javascript side, where people use promises a lot for async, but it's almost impossible to trace execution in the debugger. It's usually immediately obvious to me that many exceptions and failure modes have been missed, but I find it difficult to reason about large promise chains and I usually have no idea where I would add more error handling. It tends towards a big ball of spaghetti. Then i…

Ah, so, async/await, while conceptually similar, isn't the same across languages. Rust's semantics here are very different than JS's. I think Rust's implementation would address all of your issues, at least conceptually.

> I don't understand the advantage of copying Javascript model with all of its caveats. I think what's happening is that people just want to run with the context they're already familiar with.

With respect, I think it's because you don't understand it in Rust yet. It was not directly copied from JS, and was certainly not due to familiarity. Read the post! It's very good :)

Re: Writing an OS in Rust: Async/Await

#103
post #71

Earlier quoted context omitted.

I disagree, I find state machines almost impossible to reason about. Has it already been through state X? Who knows. Will it come back to this state in the future? Maybe. Is there a route from state Y to state Z? Shrug. State machines are effectively goto writ large, and there's a reason we switched to structured programming.

I wish I could upvote you 100. State machines are fine up to some number of states, on the order of 10. Beyond that, they become incredibly difficult to reason about in real programs because each state may have side effects with program state. What ends up happening is that exceptional situations happen and the state machine becomes muddled with special cases. So for example, say you're writing a networked game of Te…

> And since async/await is more conceptually equivalent to a state machine than a coroutine or sync/blocking code, that severely limits how well we can reason about large promise chains.

I have to disagree with that. You can implement async/await using state machines, just as you can implement a loop using goto, but it's a more constrained model that's much easier to reason about than a fully general state machine or a fully general coroutine. The control flow still proceeds the way you expect - you still execute code from top to bottom, you still return once from every function call - you just yield at some points in it. You're not interleaving your control flow with some other function that you communicate with (like a generator), and you're certainly not treating suspended control flow as something to be copied or passed around (like a coroutine). It's a much simpler concept.

Re: Writing an OS in Rust: Async/Await

#104
post #81

Aside from the cooperative multitasking discussion, I thoroughly enjoyed how you discussed the rust async/await concepts in detail and learned a lot from it. I find your article providing a lot more value than the rust documentation, which is unfortunate since that makes the learning curve somewhat difficult.

Great to hear that! I also found the official documentation a bit short on background information, so I decided to write my own explanation rather than link to something existing. Given that the async/await implementation is still quite young, I'm sure that the official docs will improve over time.

Re: Writing an OS in Rust: Async/Await

#105
post #15

Is async/await a good idea for an OS kernel, even a toy one? Cooperative multitasking tends to break down at scale, because the probability that all the "threads" you're cooperating with are playing nice goes to zero as the number of threads increases. An OS will tend to have a concentrated number of the pathological cases in it as it deals with hardware and all the other hardest timing & concurrency problems. It's a…

Probably not at all related to the OP. But I found the thought entertaining, so offering it as such. One way to address you concern would be to guarantee that task always terminate in a bounded amount of instructions. One way to solve that would be offer a limited buffer in which the program instructions for the task might be expressed, and a language to express them in that is strongly normalizing (think something l…

The seL4 microkernel, written in C, has formal proofs of the maximum clock cycles for all of its syscalls. That's one way they manage to not need in-kernel preemption.

Re: Writing an OS in Rust: Async/Await

#106
post #6

Is it possible to context switch between userspace processes directly, without going through the kernel, i.e. a kind of fast, inter-process cooperative multitasking? I know earlier operating systems used inter-process cooperative multitasking, but I'm guessing they still went through the scheduler? I was trying to figure out if QNX does this with `MsgSend`, as QNX is renowned for being a fast microkernel, but it wasn…

In the vaporware Mill CPU design, yes. On x86, afaik hard no.

Re: Writing an OS in Rust: Async/Await

#107
post #85
post #82

Earlier quoted context omitted.

> What do you think about hierarchical state machine [1] ? Would need to see what the implementation actually looked like and how it was worked on. The graphs may look nice, but I've never seen people program effectively by editing graphs. > Mentioned in the comment before, there is a transition graph that you can do static analysis on, note that you might run into a halting problem Sure, but the graph seems to be ve…

Interesting points, thanks > Would need to see what the implementation actually looked like and how it was worked on Examples are but not limited to Simulink, Rhapsody, Unreal Blueprint, QT SCXML >The graphs may look nice, but I've never seen people program effectively by editing graphs graph mentioned is a state transition graph, not necessarily a visual graph, although its a nice eventuality > You don't step throug…

> I have to agree with you on locality in pure state machine, but if a state machine is defined as an Actor you get locality by definition.

Not really. You have a kind of binary version of locality - whether something is in the same actor or not - but you can't reason about any region that's smaller or larger than that. Incoming messages might arrive from anywhere, and emitted messages might cause any kind of effect anywhere (including other messages back to the sending actor). So you can't really understand any region larger than an actor without having to understand the whole system - and that's before we even get to passing actor addresses around.

Post reply on HN