Writing an OS in Rust: Async/Await • Hacker News Board

Re: Writing an OS in Rust: Async/Await

#31

Post by retrac » Mon, Mar 30, 2020, 5:20 PM UTC

m0th87 wrote: post #6

Is it possible to context switch between userspace processes directly, without going through the kernel, i.e. a kind of fast, inter-process cooperative multitasking? I know earlier operating systems used inter-process cooperative multitasking, but I'm guessing they still went through the scheduler? I was trying to figure out if QNX does this with `MsgSend`, as QNX is renowned for being a fast microkernel, but it wasn…

It's not possible without specific hardware support.

Context-switching on most current platforms with virtual memory requires fiddling with things like page mappings, and that simply must be done from supervisor mode.

The ability to switch tasks in hardware has existed on some historical machines. For example the GEC 4000 series basically implemented a microkernel, including the scheduler, in hardware. Switching to another process was done with a single hardware instruction.

I don't think anything current has such features besides the long-obsolete and unused taskswitching feature on x86 processors.

Re: Writing an OS in Rust: Async/Await

#32

Post by pjmlp » Mon, Mar 30, 2020, 5:28 PM UTC

jerf wrote: post #15

Is async/await a good idea for an OS kernel, even a toy one? Cooperative multitasking tends to break down at scale, because the probability that all the "threads" you're cooperating with are playing nice goes to zero as the number of threads increases. An OS will tend to have a concentrated number of the pathological cases in it as it deals with hardware and all the other hardest timing & concurrency problems. It's a…

Yes, Active Oberon already had it in the mid 90's.

Symbian also had it, both called them active objects.

Re: Writing an OS in Rust: Async/Await

#33

Post by bitwize » Mon, Mar 30, 2020, 5:30 PM UTC

jerf wrote: post #15

Is async/await a good idea for an OS kernel, even a toy one? Cooperative multitasking tends to break down at scale, because the probability that all the "threads" you're cooperating with are playing nice goes to zero as the number of threads increases. An OS will tend to have a concentrated number of the pathological cases in it as it deals with hardware and all the other hardest timing & concurrency problems. It's a…

All I/O is asynchronous. Or rather, synchronous I/O is a subset of asynchronous. Often, in kernel I/O systems, you are communicating with, say, a disk controller that will take some time to process your request. You will want to do other stuff while the device finishes and process the results only when the device signals success or failure.

Re: Writing an OS in Rust: Async/Await

#34

Post by dfox » Mon, Mar 30, 2020, 5:31 PM UTC

amelius wrote: post #4

An OS with async/await sounds awfully similar to Windows 3.1 with its cooperative multitasking model. What's old is new again ...

There is one significant difference: Windows did context-switches in the blocking calls and did not rely on the program code having "the right structure" needed for straight co-routines to work.

The difference between preemptive and cooperative multitasking is not whether you do full context switches, but whether there is a way to do context switch at a point where the process does not expect it (ie. by handling some kind of timer interrupt as scheduling opportunity).

Re: Writing an OS in Rust: Async/Await

#35

Post by Rusky » Mon, Mar 30, 2020, 5:41 PM UTC

jerf wrote: post #15

Is async/await a good idea for an OS kernel, even a toy one? Cooperative multitasking tends to break down at scale, because the probability that all the "threads" you're cooperating with are playing nice goes to zero as the number of threads increases. An OS will tend to have a concentrated number of the pathological cases in it as it deals with hardware and all the other hardest timing & concurrency problems. It's a…

I'm not a Linux kernel developer, but it does have a few similar mechanisms AFAIU. If a driver needs to perform more work in response to an interrupt than would fit into the actual handler, it can use cooperatively scheduled "tasklets" or "softirqs."

Re: Writing an OS in Rust: Async/Await

#36

Post by [deleted] » Mon, Mar 30, 2020, 5:51 PM UTC

Bigpet wrote: post #26

AnIdiotOnTheNet wrote: post #14

Earlier quoted context omitted.

I'm not convinced that's actually true though, is what I mean. The big drawback of cooperative is just that a single process can monopolize resources, which in the singlecore days mean the system became unresponsive and you had no way to recover from it. That isn't really true now with multicore. As long as the OS has a core to work on it can terminate or otherwise deal with misbehaving processes. Again, this is base…

> Error: you cannot open another Chrome tab because all your cores are already used up by Slack and VSCode You would probably still want to preempt, because you're not going to rewrite all the widely used software to actually yield. Because that's the thing about cooperative multithreading, the participating parties need to cooperate. And if you look at the amount of threads that some software open it's just crazy. L…

[deleted]

Re: Writing an OS in Rust: Async/Await

#37

Post by amelius » Mon, Mar 30, 2020, 5:56 PM UTC

dfox wrote: post #34

amelius wrote: post #4

An OS with async/await sounds awfully similar to Windows 3.1 with its cooperative multitasking model. What's old is new again ...

There is one significant difference: Windows did context-switches in the blocking calls and did not rely on the program code having "the right structure" needed for straight co-routines to work. The difference between preemptive and cooperative multitasking is not whether you do full context switches, but whether there is a way to do context switch at a point where the process does not expect it (ie. by handling some…

Does Rust allow a computational for-loop to be interrupted somehow?

Computation can also be viewed as a blocking operation.

Re: Writing an OS in Rust: Async/Await

#38

Post by steveklabnik » Mon, Mar 30, 2020, 5:57 PM UTC

amelius wrote: post #37

dfox wrote: post #34

Earlier quoted context omitted.

There is one significant difference: Windows did context-switches in the blocking calls and did not rely on the program code having "the right structure" needed for straight co-routines to work. The difference between preemptive and cooperative multitasking is not whether you do full context switches, but whether there is a way to do context switch at a point where the process does not expect it (ie. by handling some…

Does Rust allow a computational for-loop to be interrupted somehow? Computation can also be viewed as a blocking operation.

The only yield points are .await points. If there's an .await in a loop, then sure, but otherwise no.

Re: Writing an OS in Rust: Async/Await

#39

Post by Matthias247 » Mon, Mar 30, 2020, 6:13 PM UTC

jerf wrote: post #15

Is async/await a good idea for an OS kernel, even a toy one? Cooperative multitasking tends to break down at scale, because the probability that all the "threads" you're cooperating with are playing nice goes to zero as the number of threads increases. An OS will tend to have a concentrated number of the pathological cases in it as it deals with hardware and all the other hardest timing & concurrency problems. It's a…

> Embedding islands of cooperative multitasking in a sea of pre-emptive multitasking seems to make a lot more sense than the other way around.

Pretty much my opinion. I think a preemptive threaded environment where some threads might run multiple async tasks (in order to boost efficiency) can make a lot of sense.

Pure cooperative environments seem too error-prone for bigger systems, because they lack the necessary isolation between tasks. Any bad task can degenerate the performance of all others tasks by blocking the execution thread.

Re: Writing an OS in Rust: Async/Await

#40

Post by animalnewbie » Mon, Mar 30, 2020, 6:14 PM UTC

zackmorris wrote: post #11

I've shied away from async/await because I haven't seen a good writeup on how to make it deterministic. Come to think of it, all of the times I've encountered it, there was no way to analyze the codepaths and prove that every exceptional situation and failure mode was handled. Maybe I missed something along the way? So my feeling about it is that it may turn out to be an evolutionary dead end, or anti-pattern at the…

I am no expert and I wish there were a start from 0 guide with examples not involving external crates but my sense is that you can write your own executor and do whatever you want. (Maybe I'm wrong?)