Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Writing an OS in Rust: Async/Await

os.phil-opp.com

Post reply on HN
Print view View original

21–30 of 107 posts

Re: Writing an OS in Rust: Async/Await

#21

Post by eximius »

jklehm wrote: post #16
steveklabnik wrote: post #5

Earlier quoted context omitted.

Well, there's a big difference: > Using async/wait, we now have basic support for cooperative multitasking in our kernel. While cooperative multitasking is very efficient, it leads to latency problems when individual tasks keep running for too long and thus prevent other tasks to run. For this reason, it makes sense to also add support for preemptive multitasking to our kernel. > In the next post, we will introduce t…

Isn't that driving against the lesson learned from the past though? 3rd party drivers can not be trusted to behave and hang the kernel when doing so with cooperative multitasking.

Toy OS, so he has less to worry about. But also maybe just be more careful around kernel modules, if that eventually exists?

Re: Writing an OS in Rust: Async/Await

#22

Post by [deleted] »

jerf wrote: post #15

Is async/await a good idea for an OS kernel, even a toy one? Cooperative multitasking tends to break down at scale, because the probability that all the "threads" you're cooperating with are playing nice goes to zero as the number of threads increases. An OS will tend to have a concentrated number of the pathological cases in it as it deals with hardware and all the other hardest timing & concurrency problems. It's a…

[deleted]

Re: Writing an OS in Rust: Async/Await

#24

Post by Koshkin »

steveklabnik wrote: post #5
amelius wrote: post #4

An OS with async/await sounds awfully similar to Windows 3.1 with its cooperative multitasking model. What's old is new again ...

Well, there's a big difference: > Using async/wait, we now have basic support for cooperative multitasking in our kernel. While cooperative multitasking is very efficient, it leads to latency problems when individual tasks keep running for too long and thus prevent other tasks to run. For this reason, it makes sense to also add support for preemptive multitasking to our kernel. > In the next post, we will introduce t…

GP is not wrong, then: with this implementation, async/await still amounts to the good old cooperative multitasking and does not (yet?) take advantage of threads. (In C#, for example, async/await is indeed different from this, being based on TAP - Task-based Asynchronous Pattern, where tasks are executed "asynchronously on a thread pool thread rather than synchronously on the main application thread.")

Re: Writing an OS in Rust: Async/Await

#25

Post by mtnygard »

jklehm wrote: post #16
steveklabnik wrote: post #5

Earlier quoted context omitted.

Well, there's a big difference: > Using async/wait, we now have basic support for cooperative multitasking in our kernel. While cooperative multitasking is very efficient, it leads to latency problems when individual tasks keep running for too long and thus prevent other tasks to run. For this reason, it makes sense to also add support for preemptive multitasking to our kernel. > In the next post, we will introduce t…

Isn't that driving against the lesson learned from the past though? 3rd party drivers can not be trusted to behave and hang the kernel when doing so with cooperative multitasking.

Phil's series is meant for learning rather than production use. So sometimes he takes a "shortcut" in one post then removes the restrictions or workarounds later.

Re: Writing an OS in Rust: Async/Await

#26

Post by Bigpet »

AnIdiotOnTheNet wrote: post #14
bluejekyll wrote: post #9

Earlier quoted context omitted.

As Steve’s sibling comment points out, this is only for the kernel. The article mentions preemptive threading for the user space as still being desirable.

I'm not convinced that's actually true though, is what I mean. The big drawback of cooperative is just that a single process can monopolize resources, which in the singlecore days mean the system became unresponsive and you had no way to recover from it. That isn't really true now with multicore. As long as the OS has a core to work on it can terminate or otherwise deal with misbehaving processes. Again, this is base…

> Error: you cannot open another Chrome tab because all your cores are already used up by Slack and VSCode

You would probably still want to preempt, because you're not going to rewrite all the widely used software to actually yield.

Because that's the thing about cooperative multithreading, the participating parties need to cooperate.

And if you look at the amount of threads that some software open it's just crazy. Looking on my machine the top5 are:

1. 260 threads System (ok, that's basically the OS that could be changed).

2. 145 threads Dropbox.exe (that's enough to lock all cores on any single consumer CPU).

3. 87 threads SearchIndexer.exe (also OS, could be rectified).

4. 74 threads EpicGamesLauncher (looks like an i9 is no longer enough)

5. 70 threads MemoryCompression (also OS)

I know not all of those threads are active at all time an I guess all the blocking threads could be said to be cooperating but even Dropbox alone regularly has 3-5 threads running.

There's still so many 2 and 4 core consumer machines out there that cooperative multi-tasking with the current software ecosystem would be a disaster.

Not only would applications keep each other from making regular steady progress but even single applications would regularly hang themselves from all the threads they themselves created if there was no pre-emptive yielding beyond using any OS-call as a yield-point

Re: Writing an OS in Rust: Async/Await

#27

Post by briangold »

jerf wrote: post #15

Is async/await a good idea for an OS kernel, even a toy one? Cooperative multitasking tends to break down at scale, because the probability that all the "threads" you're cooperating with are playing nice goes to zero as the number of threads increases. An OS will tend to have a concentrated number of the pathological cases in it as it deals with hardware and all the other hardest timing & concurrency problems. It's a…

True for a general-purpose OS kernel. But in an embedded system or unikernel it could make a lot of sense.

Re: Writing an OS in Rust: Async/Await

#28

Post by steveklabnik »

Koshkin wrote: post #24
steveklabnik wrote: post #5

Earlier quoted context omitted.

Well, there's a big difference: > Using async/wait, we now have basic support for cooperative multitasking in our kernel. While cooperative multitasking is very efficient, it leads to latency problems when individual tasks keep running for too long and thus prevent other tasks to run. For this reason, it makes sense to also add support for preemptive multitasking to our kernel. > In the next post, we will introduce t…

GP is not wrong, then: with this implementation, async/await still amounts to the good old cooperative multitasking and does not (yet?) take advantage of threads. (In C#, for example, async/await is indeed different from this, being based on TAP - Task-based Asynchronous Pattern, where tasks are executed "asynchronously on a thread pool thread rather than synchronously on the main application thread.")

Async/await lets you build tasks, but is completely orthogonal to threads. You could set up tasks with a 1:1 or N:M or even single threaded model.

And yes, the GP is not wrong that if this were the only mechanism provided, it would be similar. It does not seem like the plan is to expose this externally whatsoever, though.

Re: Writing an OS in Rust: Async/Await

#29

Post by ww520 »

jerf wrote: post #15

Is async/await a good idea for an OS kernel, even a toy one? Cooperative multitasking tends to break down at scale, because the probability that all the "threads" you're cooperating with are playing nice goes to zero as the number of threads increases. An OS will tend to have a concentrated number of the pathological cases in it as it deals with hardware and all the other hardest timing & concurrency problems. It's a…

An OS has many aspects to it. Scheduling tasks is only one of the aspects. Async/await is just the interface/mechanism in dealing with the tasks, in this case cooperative tasks.

Interacting with hardware, dealing with interrupts, memory mapping/managing, task isolation, etc are all other aspects of an OS that are apart from task scheduling but still needed.

Cooperative multitasking works fine as long as the users/developers understand the limitation.

Re: Writing an OS in Rust: Async/Await

#30

Post by ww520 »

zackmorris wrote: post #11

I've shied away from async/await because I haven't seen a good writeup on how to make it deterministic. Come to think of it, all of the times I've encountered it, there was no way to analyze the codepaths and prove that every exceptional situation and failure mode was handled. Maybe I missed something along the way? So my feeling about it is that it may turn out to be an evolutionary dead end, or anti-pattern at the…

Async or parallel systems are deterministic when the multiple paths don't cross, and when crossed the crossing points have well defined deterministic steps and interaction.
Post reply on HN