Aside from the cooperative multitasking discussion, I thoroughly enjoyed how you discussed the rust async/await concepts in detail and learned a lot from it. I find your article providing a lot more value than the rust documentation, which is unfortunate since that makes the learning curve somewhat difficult.
Writing an OS in Rust: Async/Await
81–90 of 107 posts
Re: Writing an OS in Rust: Async/Await
#82Earlier quoted context omitted.
I disagree, I find state machines almost impossible to reason about. Has it already been through state X? Who knows. Will it come back to this state in the future? Maybe. Is there a route from state Y to state Z? Shrug. State machines are effectively goto writ large, and there's a reason we switched to structured programming.
What do you think about hierarchical state machine [1] ? >Has it already been through state X? Like everything else, logging >Will it come back to this state in the future? Mentioned in the comment before, there is a transition graph that you can do static analysis on, note that you might run into a halting problem >Is there a route from state Y to state Z? There is a transition graph >State machines are effectively…
> What do you think about hierarchical state machine [1] ?
Would need to see what the implementation actually looked like and how it was worked on. The graphs may look nice, but I've never seen people program effectively by editing graphs.
> Mentioned in the comment before, there is a transition graph that you can do static analysis on, note that you might run into a halting problem
Sure, but the graph seems to be very much secondary. You don't step through the graph in a debugger, for example.
> quoting Prof Carl Hewitt of Actor Model, "goto is harmless"[2]
Not at all convinced, and he fails to make any real case for it. I've worked on actor systems, and they're bad in the same way that unstructured code is bad.
> function/procedure call is a goto,
It's not, because you still have the call stack. You know where you came from and why (and can see it in the debugger), and you know you're going back there. You can reason compositionally, because in g(f(x)) f is a black box and always will be even if it gets refactored, whereas there's no equivalent "locality" in a state machine.
> much like sending an named event while in particular state with/out parameters
That also sounds like a bad way of doing things.
Re: Writing an OS in Rust: Async/Await
#83Earlier quoted context omitted.
I disagree, I find state machines almost impossible to reason about. Has it already been through state X? Who knows. Will it come back to this state in the future? Maybe. Is there a route from state Y to state Z? Shrug. State machines are effectively goto writ large, and there's a reason we switched to structured programming.
Well-designed deterministic state machines are incredibly easy to reason about. You simply render them as a graph and at a glance you can understand every possible transition and state. > Will it come back to this state in the future? If there's a path from the current state back to itself and it gets the correct inputs, yes. Otherwise no. > Is there a route from state Y to state Z? A simple glance at the graph will…
> You simply render them as a graph and at a glance you can understand every possible transition and state.
But programming by editing graphs never caught on. So the graph is always going to be a secondary representation of your program; it won't be the natural view in your debugger or profiler (if it's even visible at all).
And more importantly, no-one's found a really compositional approach to programming with graphs yet. In conventional programming you can define an interface that exposes one function, and maybe you have an incredibly complex implementation behind that interface, but you can reason about the rest of your program without having to look inside that black box, even as the implementation changes drastically. I've never seen the same approach applied to state machines: maybe you know that this cluster of states is independent now, but anyone can add a new state transition that changes the global control flow willy-nilly.
Re: Writing an OS in Rust: Async/Await
#84Is async/await a good idea for an OS kernel, even a toy one? Cooperative multitasking tends to break down at scale, because the probability that all the "threads" you're cooperating with are playing nice goes to zero as the number of threads increases. An OS will tend to have a concentrated number of the pathological cases in it as it deals with hardware and all the other hardest timing & concurrency problems. It's a…
> Is async/await a good idea for an OS kernel, even a toy one? You might want to look at Joe Duffy's posts about Microsoft's Midori OS. In particular the post "Asynchronous Everything": http://joeduffyblog.com/2015/11/03/blogging-about-midori/ Their findings sound pretty compelling to me. Personally I'm convinced that eventually we'll see much more system-level use of async/await style mechanisms. Perhaps with Rust,…
You can already use C++ coroutines with C++/Winrt, if feeling like having a go at them.
Re: Writing an OS in Rust: Async/Await
#85Earlier quoted context omitted.
What do you think about hierarchical state machine [1] ? >Has it already been through state X? Like everything else, logging >Will it come back to this state in the future? Mentioned in the comment before, there is a transition graph that you can do static analysis on, note that you might run into a halting problem >Is there a route from state Y to state Z? There is a transition graph >State machines are effectively…
> What do you think about hierarchical state machine [1] ? Would need to see what the implementation actually looked like and how it was worked on. The graphs may look nice, but I've never seen people program effectively by editing graphs. > Mentioned in the comment before, there is a transition graph that you can do static analysis on, note that you might run into a halting problem Sure, but the graph seems to be ve…
Interesting points, thanks
> Would need to see what the implementation actually looked like and how it was worked on
Examples are but not limited to Simulink, Rhapsody, Unreal Blueprint, QT SCXML
>The graphs may look nice, but I've never seen people program effectively by editing graphs
graph mentioned is a state transition graph, not necessarily a visual graph, although its a nice eventuality
> You don't step through the graph in a debugger, for example.
Yes you do, if using the mentioned example
Combining a state machine with actor model has been a work wonder for me. I have to agree with you on locality in pure state machine, but if a state machine is defined as an Actor you get locality by definition. Its more of a deterministic abstract modelling, testing and execution
Re: Writing an OS in Rust: Async/Await
#86Is async/await a good idea for an OS kernel, even a toy one? Cooperative multitasking tends to break down at scale, because the probability that all the "threads" you're cooperating with are playing nice goes to zero as the number of threads increases. An OS will tend to have a concentrated number of the pathological cases in it as it deals with hardware and all the other hardest timing & concurrency problems. It's a…
>Cooperative multitasking tends to break down at scale, because the probability that all the "threads" you're cooperating with are playing nice goes to zero as the number of threads increases.
That's completely wrong. Either cooperative multitasking works or it doesn't. There is no probability involved. What often happens is that foreign code that is not under your control is not yielding. It only takes a single non cooperating task to block an entire core. If you have properly written code the probability of a malfunction is 0% and this probability doesn't grow with the number of threads.
Re: Writing an OS in Rust: Async/Await
#87An OS with async/await sounds awfully similar to Windows 3.1 with its cooperative multitasking model. What's old is new again ...
Just because a kernel is using something internally doesn't mean it is exposing it to user space applications.
Re: Writing an OS in Rust: Async/Await
#88Earlier quoted context omitted.
I'm not convinced that's actually true though, is what I mean. The big drawback of cooperative is just that a single process can monopolize resources, which in the singlecore days mean the system became unresponsive and you had no way to recover from it. That isn't really true now with multicore. As long as the OS has a core to work on it can terminate or otherwise deal with misbehaving processes. Again, this is base…
> Error: you cannot open another Chrome tab because all your cores are already used up by Slack and VSCode You would probably still want to preempt, because you're not going to rewrite all the widely used software to actually yield. Because that's the thing about cooperative multithreading, the participating parties need to cooperate. And if you look at the amount of threads that some software open it's just crazy. L…
The intention of the author's OS is to use cooperation in kernel space and preemption in user space so your entire comment makes absolutely no sense.
Re: Writing an OS in Rust: Async/Await
#89Earlier quoted context omitted.
"Now do note that this requires a far more complex executor model than most async/await toolkits (including this tutorial) provide." I'm assuming the async/await language support in Rust is intrinsically tied to a cooperative approach, which is the part I'm questioning. Obviously a kernel needs to be able to generically work in an asynchronous fashion... the question is, is this asynchronous fashion appropriate? If t…
> But a fundamentally cooperative kernel would really worry me. You'd be looking at freezing the whole kernel, and likely the whole machine, if anything, anywhere, goes wrong with the cooperation. It won't take a lot of scale before that's a problem. This is already the case, isn’t it? If you do a ‘while(!operation_complete) {}’ anywhere in the kernel and it never happens then I’d expect it’s pretty much done. Wherea…
The parallel case with async functions should be a long running computation that does not contain any yield point, so that it cannot be suspended easily. I remember that it is possible to construct (contrived) examples of this also in Erlang.
Re: Writing an OS in Rust: Async/Await
#90Earlier quoted context omitted.
But if Rust is the operating system/kernel, whenever it decides to to schedule something is preemption for anything downstream, right? I mean, you don't actually use preemption in the kernel right? Don't you have to handle all that yourself, since there's nothing higher level than you to handle it for you? In that respect, doesn't plugging in a Futures runtime that looks for device flags and stuff as appropriate and…
If you would write a basic scheduler, at some point you'd have to await the userspace code but you wouldn't have any way to force it to stop running. If the userspace code would enter an infinite loop it would hold the kernel thread forever. Within a constrained environment, eg. the kernel itself (and even that's sufficiently complex with loadable drivers that you might end up with bad interactions) I could see some…
As far as I understand the scheduler and userspace processes are going to be completely orthogonal features to this. In general would it be complex[1] in this case to integrate both preemptive and cooperative multitasking?
[1] as in: http://journal.stuffwithstuff.com/2015/02/01/what-color-is-y...