Live data from Hacker News

Monoio – A thread-per-core Rust async runtime with io_uring

github.com

71–80 of 84 posts

Re: Monoio – A thread-per-core Rust async runtime with io_uring

#71

Earlier quoted context omitted.

I am a Windows user, so all of the Rust software I use runs on it. And that’s virtually all Rust software. Sometimes you need a small patch or two because someone did something weird with path handling, but 99.99% of it Just Works. Apparently we haven't published platform statistics since 2019, but according to that years' survey: https://blog.rust-lang.org/images/2020-03-RustSurvey/32-what... * 55% of Rust users dev…

I'm only saying it's a tradeoff. Obviously given Rust's origins and goals dropping desktop as a target would make no sense - it was designed explicitly for those platforms. But if a language said "we're not going to support those" I wouldn't care at all, and a massive number of use cases - the majority, I think - would be solved with that language. In terms of what you develop on , that's a whole other story. The maj…

> In terms of what you develop on, that's a whole other story. The majority, of course, are on Linux

I'm not sure that's as obvious as you're making it sound. The number of people who use Linux on the desktop is absolutely minuscule compared to the combined user base of Windows and MacOS. It's probably not as lopsided for developers, but I've never seen anything to imply that most developers in general are on Linux, and I'd honestly be surprised if that's the case given how much smaller the portion is in terms of people I know or have worked with, and that's as someone who does not personally own any laptop or desktop that runs anything other than Linux.

Re: Monoio – A thread-per-core Rust async runtime with io_uring

#72
post #71

Earlier quoted context omitted.

I'm only saying it's a tradeoff. Obviously given Rust's origins and goals dropping desktop as a target would make no sense - it was designed explicitly for those platforms. But if a language said "we're not going to support those" I wouldn't care at all, and a massive number of use cases - the majority, I think - would be solved with that language. In terms of what you develop on , that's a whole other story. The maj…

> In terms of what you develop on, that's a whole other story. The majority, of course, are on Linux I'm not sure that's as obvious as you're making it sound. The number of people who use Linux on the desktop is absolutely minuscule compared to the combined user base of Windows and MacOS. It's probably not as lopsided for developers, but I've never seen anything to imply that most developers in general are on Linux,…

The linked poll has the majority of rust developers on linux

Re: Monoio – A thread-per-core Rust async runtime with io_uring

#73

Earlier quoted context omitted.

A cooperative synchronous task switching (i.e. fiber based) need only save the exact same information as an async based one (i.e. stackless coroutines): at a minimum a context pointer and an instruction pointer. Plus any live registers (which might be none). You only need to save caller saved registers fs your context switching routine uses the conventional ABI, but that's not a requirement.

> You only need to save caller saved registers fs your context switching routine uses the conventional ABI, but that's not a requirement. If you request a task switch from C or any high level language that has the concept of caller-saved registers and the compiler has no knowledge of your task switching system (vast majority of cases) you will be forced to pay an extra cost. Is there a practical system in common use…

The assumption is that you control the compiler (we are comparing with async that outside of duff device hacks does require compiler help).

But even without explicit compiler help, you can go a long way, say, with gcc extended inline asm.

Re: Monoio – A thread-per-core Rust async runtime with io_uring

#74

Earlier quoted context omitted.

> You only need to save caller saved registers fs your context switching routine uses the conventional ABI, but that's not a requirement. If you request a task switch from C or any high level language that has the concept of caller-saved registers and the compiler has no knowledge of your task switching system (vast majority of cases) you will be forced to pay an extra cost. Is there a practical system in common use…

The assumption is that you control the compiler (we are comparing with async that outside of duff device hacks does require compiler help). But even without explicit compiler help, you can go a long way, say, with gcc extended inline asm.

You don’t need explicit compiler help. At least with C this can be done entirely with a library. A task_switch() call can conform to the standard C-ABI, requiring no compiler support, and do the switching (in assembly). Without duff’s device. This is for example how kernels written in C do their task switching.

Likely the same can be said for Rust and nearly any language, since they all have ABIs to which can be conformed such that task_switch() looks like a normal function call.

Re: Monoio – A thread-per-core Rust async runtime with io_uring

#75

Earlier quoted context omitted.

The assumption is that you control the compiler (we are comparing with async that outside of duff device hacks does require compiler help). But even without explicit compiler help, you can go a long way, say, with gcc extended inline asm.

You don’t need explicit compiler help. At least with C this can be done entirely with a library. A task_switch() call can conform to the standard C-ABI, requiring no compiler support, and do the switching (in assembly). Without duff’s device. This is for example how kernels written in C do their task switching. Likely the same can be said for Rust and nearly any language, since they all have ABIs to which can be conf…

Oh, I have written my own share of userspace C context switching libraries, I know all the gory the details :). For example see my minimalist [1] stackful coroutine library: the full context switching logic is three inline asm instructions (99% of the complexity in that code is to transparently support throwing exceptions across coroutine boundaries with no overhead in the happy path).

You need compiler help for the custom calling convention support and possibly to optimize away the context switching overhead for stackful coroutines, which is something that compilers can already do for stackless coroutines.

The duff device is just a way to simulate stackless coroutines (i.e. async/await or whateverer) in plain C, in a way that the compiler can still optimize quite well.

[1] https://github.com/gpderetta/delimited/blob/master/delimited...

Re: Monoio – A thread-per-core Rust async runtime with io_uring

#76

Earlier quoted context omitted.

> It's truly understated how difficult the Send + Sync requirements in Tokio are for writing regular code. It's typically rare for async tasks in Tokio to be used across two threads simultaneously, but now all of your data must be Send+Sync. https://docs.rs/tokio/1.14.0/tokio/task/fn.spawn_local.html > Plus, Tokio is unique in that it's one of the very few runtimes in existence that's work-stealing (meaning the task…

Quoting from a reply on an issue in the rust async-wg repo: > Are people doing this? Does it work? :) > Kind of every single successful async framework since the inception of eventloops :-) > libevent, libev, libuv, boost asio, GTK, QT, seastar, nginx, javascript + node.js, dpdk, netty, grizzly, dart, etc. > Besides Rust the main frameworks which tried to do move tasks between executors are Go and C#'s Threadpool exe…

I have absolutely no idea what the relevance of that mess is.

Are you arguing rust’s async runtimes should be single threaded, right after having somehow expressed interest in a non-single-threaded runtime?

Re: Monoio – A thread-per-core Rust async runtime with io_uring

#77
post #71

Earlier quoted context omitted.

> In terms of what you develop on, that's a whole other story. The majority, of course, are on Linux I'm not sure that's as obvious as you're making it sound. The number of people who use Linux on the desktop is absolutely minuscule compared to the combined user base of Windows and MacOS. It's probably not as lopsided for developers, but I've never seen anything to imply that most developers in general are on Linux,…

The linked poll has the majority of rust developers on linux

Sure, but only barely a majority. Nearly half of Rust users are on Windows or MacOS. Dropping support for those would be crazily irresponsible and would probably be a bigger programming scandal than even the Python 2->3 ordeal. And for what benefit? Making some low level libraries easier?

Re: Monoio – A thread-per-core Rust async runtime with io_uring

#78

Earlier quoted context omitted.

You don’t need explicit compiler help. At least with C this can be done entirely with a library. A task_switch() call can conform to the standard C-ABI, requiring no compiler support, and do the switching (in assembly). Without duff’s device. This is for example how kernels written in C do their task switching. Likely the same can be said for Rust and nearly any language, since they all have ABIs to which can be conf…

Oh, I have written my own share of userspace C context switching libraries, I know all the gory the details :). For example see my minimalist [1] stackful coroutine library: the full context switching logic is three inline asm instructions (99% of the complexity in that code is to transparently support throwing exceptions across coroutine boundaries with no overhead in the happy path). You need compiler help for the…

> the full context switching logic is three inline asm instructions

You tell the compiler that you clobber the caller-saved registers (GPD_CLOBBERS), so in terms of cost it’s not just three asm instructions. Since these are caller-saved registers they will be live at every point in your code, even if your task switch routine is inlined. You have to consider the code the compiler generates to preserve the caller-saved registers before invoking your instruction sequence when evaluating total cost. This is an additional cost that is not necessary in callback style.

Re: Monoio – A thread-per-core Rust async runtime with io_uring

#79

Earlier quoted context omitted.

Oh, I have written my own share of userspace C context switching libraries, I know all the gory the details :). For example see my minimalist [1] stackful coroutine library: the full context switching logic is three inline asm instructions (99% of the complexity in that code is to transparently support throwing exceptions across coroutine boundaries with no overhead in the happy path). You need compiler help for the…

> the full context switching logic is three inline asm instructions You tell the compiler that you clobber the caller-saved registers (GPD_CLOBBERS), so in terms of cost it’s not just three asm instructions. Since these are caller-saved registers they will be live at every point in your code, even if your task switch routine is inlined. You have to consider the code the compiler generates to preserve the caller-saved…

Caller-saved registers (aka. "volatile registers") are only saved when they are live in the caller at the point of a function call, and they are not always live. Code generation tends to prefer callee-saved registers instead at these points, precisely so they don't need to be saved there. Whether callee-saved registers are live at the inline task switch depends on whether they have been saved already in the function prologue, and if they are in use as temporaries. Not many registers are live at every point, typically just the stack and frame pointers.

Both types of code (async and stackful) have to save live state, whetever it is, across context switches, whether that's spilling registers to the stack in the stackful case, or into the future object across "await" in the async case. However, typically the code generator has more leeway to decide which spills are optimal in the stackful case, and the spills are to the hot stack, so low cost. Unless integrated with the code generator, async/await spills tend to be unconditional stores and loads, thus on average more expensive.

You're right about potentially redundant saves at stackful context switch. (Though if you control the compiler, as you should if you are comparing best implementations of both kinds, you can avoid truly redundant saves)

However, in practice few of the callee-saved registers are really redundant. If the caller doesn't use them, it's caller or some ancestor further up the chain usually does. If any do, they are genuine live state rather than redundant. There are cases you can construct where no ancestor uses a register, or uses one when it would be better not to, so that in theory it would be better not to use it and not to save it on context switch. But I think this is rare in real code.

You must compare this against the the various extra state storage, and memory allocations, in async/await: For example storing results in future objects in some implementations, spilling live state to an object when the stackful compiler would have used a register or the hot stack, and the way async/await implementations tend to allocate, fill, and later free a separate future object for each level of await in the call stack. All that extra storing is not free. Also, when comparing against the best of stackful, how many await implementations compile to pure continuation jumps without a return to an event loop function just to call the next async handler, and how many allow await results to be transferred directly in registers from generator to consumer, without being stored in the allocated future object?

I would summarise the difference between async/await and stackful-cooperative is that the former has considerable memory op overheads, but they are diffused throughout the code, so the context switch itself looks simple. It's an illusion, though, just like the "small asm" stackful context switch is an illusion due to clobbered live registers. The overhead is still there, either way, and I think it's usually slightly higher overhead in the async/await version. But async/await does have the advantage of not needing a fixed size "large enough for anything" stack to be preallocated per context, which it replaces with multiple and ongoing smaller allocations & frees per context instead.

It would be interesting to see an async/await transform applied to the Linux kernel, to see if it ended up faster or slower.

Re: Monoio – A thread-per-core Rust async runtime with io_uring

#80
post #79

Earlier quoted context omitted.

> the full context switching logic is three inline asm instructions You tell the compiler that you clobber the caller-saved registers (GPD_CLOBBERS), so in terms of cost it’s not just three asm instructions. Since these are caller-saved registers they will be live at every point in your code, even if your task switch routine is inlined. You have to consider the code the compiler generates to preserve the caller-saved…

Caller-saved registers (aka. "volatile registers") are only saved when they are live in the caller at the point of a function call, and they are not always live. Code generation tends to prefer callee -saved registers instead at these points, precisely so they don't need to be saved there. Whether callee-saved registers are live at the inline task switch depends on whether they have been saved already in the function…

Thanks for the great contribution to the thread. Pretty much my thoughts.

I do believe that having to allocate a large stack is a downside, but again, with compiler help it should be possible, at least in theory, to compile stackful coroutines whose stack usage is known and bounded (i.e all task switch happen at top level or on non recursive inlined functions) to exactly the same stack usage of stackless coroutines.

Post reply on HN