Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Io_uring, kTLS and Rust for zero syscall HTTPS server

blog.habets.se

Post reply on HN
Print view View original

81–90 of 173 posts

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#81

Post by oconnor663 »

newpavlov wrote: post #70
duped wrote: post #67

Earlier quoted context omitted.

That problem exists regardless of whether you want to use stackful coroutines or not. The stack could be freed by user code at anytime. It could also panic and drop buffers upon unwinding. I wouldn't call async drop a pile of hacks, it's actually something that would be useful in this context. And that said there's an easy fix: don't use the pointers supplied by the future!

>That problem exists regardless of whether you want to use stackful coroutines or not. The stack could be freed by user code at anytime. It could also panic and drop buffers upon unwinding. Nope. The problem does not exist in the stackfull model by the virtue of user being unable (in safe code) to drop stack of a stackfull task similarly to how you can not drop stack of a thread. If you want to cancel a stackfull tas…

> don't use `let mut buf = [0u8; 16]; socket.read_all(&mut buf).await?;`. If you can't see why such arguments are bonkers, we don't have anything left to talk about.

It doesn't seem bonkers to me. I know you already know these details, but spelling it out: If I'm using select/poll/epoll in C to do non-blocking reads of a socket, then yes I can use any old stack buffer to receive the bytes, because those are readiness APIs that only write through my pointer "now or never". But if I'm using IOCP/io_uring, I have to be careful not to use a stack buffer that doesn't outlive the whole IO loop, because those are completion APIs that write through my pointer "later". This isn't just a question of the borrow checker being smart enough to analyze our code; it's a genuine difference in what correct code needs to do in these two different settings. So if async Rust forces us to use heap allocated (or long-lived in some other way) buffers to do IOCP/io_uring reads, is that a failure of the async model, or is that just the nature of systems programming?

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#82

Post by vlovich123 »

newpavlov wrote: post #65
duped wrote: post #62

Earlier quoted context omitted.

> You simply can not do it with state machines polled in user code That's not really true. The only guarantees in Rust futures are that they are polled() once and must have their Waker's wake() called before they are polled again. A completion based future submits the request on first poll and calls wake() on completion. That's kind of the interesting design of futures in Rust - they support polling and completion. T…

No, the fundamental problem (in the context of io-uring) is that futures are managed by user code and can be dropped at any time. This often referred as "cancellation safety". Imagine a future has initialized completion-based IO with buffer which is part of the future state. User code can simply drop the future (e.g. if it was part of `select!`) and now we have a huge problem on our hands: the kernel will write into…

I really don’t understand this argument. If you force the user to transfer ownership of the buffer into the I/O subsystem, the system can make sure to transfer ownership of the buffer into the async runtime, not leaving it held within the cancellable future and the future returns that buffer which is given back when the completion is received from the kernel. What am I missing?

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#83

Post by duped »

newpavlov wrote: post #70
duped wrote: post #67

Earlier quoted context omitted.

That problem exists regardless of whether you want to use stackful coroutines or not. The stack could be freed by user code at anytime. It could also panic and drop buffers upon unwinding. I wouldn't call async drop a pile of hacks, it's actually something that would be useful in this context. And that said there's an easy fix: don't use the pointers supplied by the future!

>That problem exists regardless of whether you want to use stackful coroutines or not. The stack could be freed by user code at anytime. It could also panic and drop buffers upon unwinding. Nope. The problem does not exist in the stackfull model by the virtue of user being unable (in safe code) to drop stack of a stackfull task similarly to how you can not drop stack of a thread. If you want to cancel a stackfull tas…

> The problem does not exist in the stackfull model by the virtue of user being unable (in safe code) to drop stack of a stackfull task similarly to how you can not drop stack of a thread.

If you're not doing things better than threads then why don't you just use threads?

> And you can not fundamentally panic while waiting for a completion event, the task code is "frozen" until the signal is received.

So you only allow join/select at the task level? Sounds awful!

> Let met translate: don't use `let mut buf = [0u8; 16]; socket.read_all(&mut buf).await?;

Yes, exactly. It's more like `let buf = socket.read(16);`

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#84

Post by thomashabets2 »

ori_b wrote: post #77
thomashabets2 wrote: post #66

Earlier quoted context omitted.

Rust: Well yes. Rust does force you to understand the things, or it won't compile. It does have drawbacks. Go: goroutines are not async. And you can't understand goroutines without understanding channels. And channels are weirdly implemented in Go, where the semantics of edge cases, while well defined, are like rolling a D20 die if you try to reason from first principles. Go doesn't force you to understand things. I…

> Go: goroutines are not async Sure they are. The abstraction they provide is a synchronous API, but it's accomplished using an async runtime.

I'm trying to understand the context in which the parent commenter uses the term, since it can mean multiple things. They said "async" and then enumerated some wildly different things.

Like, do you need async runtimes to do epoll async in Rust? No. Ok, so that excludes many definitions. Do you need coroutines in C++ to do aio for reading and writing? No.

So like I said, what do they mean by "async"? The blog post refers to a web server that does "async" in Rust without any async runtime, and without the `async` keyword.

In other words, that parent commenter is what's called "not even wrong".

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#85

Post by newpavlov »

vlovich123 wrote: post #82
newpavlov wrote: post #65

Earlier quoted context omitted.

No, the fundamental problem (in the context of io-uring) is that futures are managed by user code and can be dropped at any time. This often referred as "cancellation safety". Imagine a future has initialized completion-based IO with buffer which is part of the future state. User code can simply drop the future (e.g. if it was part of `select!`) and now we have a huge problem on our hands: the kernel will write into…

I really don’t understand this argument. If you force the user to transfer ownership of the buffer into the I/O subsystem, the system can make sure to transfer ownership of the buffer into the async runtime, not leaving it held within the cancellable future and the future returns that buffer which is given back when the completion is received from the kernel. What am I missing?

The goal of the async system is to allow users to write synchronous looking code which is executed asynchronously with all associated benefits. "Forcing" users to do stuff like this shows the clear failure to achieve this goal. Additionally, passing ownership like this (instead of passing mutable borrow) arguably goes against the zero-cost principle.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#86

Post by vlovich123 »

newpavlov wrote: post #85
vlovich123 wrote: post #82

Earlier quoted context omitted.

I really don’t understand this argument. If you force the user to transfer ownership of the buffer into the I/O subsystem, the system can make sure to transfer ownership of the buffer into the async runtime, not leaving it held within the cancellable future and the future returns that buffer which is given back when the completion is received from the kernel. What am I missing?

The goal of the async system is to allow users to write synchronous looking code which is executed asynchronously with all associated benefits. "Forcing" users to do stuff like this shows the clear failure to achieve this goal. Additionally, passing ownership like this (instead of passing mutable borrow) arguably goes against the zero-cost principle.

I don’t follow the zero copy argument. You pass in an owned buffer and get an owned buffer back out. There’s no copying happening here. It’s your claim that async is supposed to look like synchronous code but I don’t buy it. I don’t see why that’s a goal. Synchronous is an anachronistic software paradigm for a computer hardware architecture that never really existed (electronics are concurrent and asynchronous by nature) and cause a lot of performance problems trying to make it work that way.

Indeed, one thing I’ve always wondered is if you can submit a read request for a page aligned buffer and have the kernel arrange for data to be written directly into that without any additional copies. That’s probably not possible since there’s routing happening in the kernel and it accumulates everything into sk_buffs.

But maybe it could arrange for the framing part of the packet and the data to be decoupled so that it can just give you a mapping into the data region (maybe instead of you even providing a buffer, it gives you back an address mapped into your space). Not sure if that TLB update might be more expensive than a single copy.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#87

Post by newpavlov »

duped wrote: post #83
newpavlov wrote: post #70

Earlier quoted context omitted.

>That problem exists regardless of whether you want to use stackful coroutines or not. The stack could be freed by user code at anytime. It could also panic and drop buffers upon unwinding. Nope. The problem does not exist in the stackfull model by the virtue of user being unable (in safe code) to drop stack of a stackfull task similarly to how you can not drop stack of a thread. If you want to cancel a stackfull tas…

> The problem does not exist in the stackfull model by the virtue of user being unable (in safe code) to drop stack of a stackfull task similarly to how you can not drop stack of a thread. If you're not doing things better than threads then why don't you just use threads? > And you can not fundamentally panic while waiting for a completion event, the task code is "frozen" until the signal is received. So you only all…

>If you're not doing things better than threads then why don't you just use threads?

Because green threads are more efficient than the classical threads. You have less context switching, more control over concurrency (e.g. you can have application-level pseudo critical section and tools like `join!`/`select!`), and with io-uring you have a much smaller number of syscalls.

In other words, memory footprint would be similar to the classical threads, but runtime performance can be much higher.

>So you only allow join/select at the task level? Sounds awful!

What is the difference with join/select at the future level?

Yes, with the most straightforward implementation you have to allocate full stack for each sub-task (somewhat equivalent to boxing sub-futures). But it's theoretically possible to use the parent task stack for sub-task stacks with the aforementioned compiler improvements.

Another difference is that instead of just dropping the future state on the floor you have to explicitly send a cancellation signal (e.g. based on `IORING_OP_ASYNC_CANCEL`) and wait for the sub-task to finish. Performance-wise it should have minimal difference when compared against the hypothetical async Drop.

>Yes, exactly.

Ok, I have nothing more to add then.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#88

Post by newpavlov »

oconnor663 wrote: post #81
newpavlov wrote: post #70

Earlier quoted context omitted.

>That problem exists regardless of whether you want to use stackful coroutines or not. The stack could be freed by user code at anytime. It could also panic and drop buffers upon unwinding. Nope. The problem does not exist in the stackfull model by the virtue of user being unable (in safe code) to drop stack of a stackfull task similarly to how you can not drop stack of a thread. If you want to cancel a stackfull tas…

> don't use `let mut buf = [0u8; 16]; socket.read_all(&mut buf).await?;`. If you can't see why such arguments are bonkers, we don't have anything left to talk about. It doesn't seem bonkers to me. I know you already know these details, but spelling it out: If I'm using select/poll/epoll in C to do non-blocking reads of a socket, then yes I can use any old stack buffer to receive the bytes, because those are readiness…

>is that a failure of the async model

This, 100%. Being really generous, it can be called a leaky model which is poorly compatible with completion-based APIs.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#89

Post by oconnor663 »

npalli wrote: post #63

So, current status on async Rust - you need to understand: Futures, Pin, Waker, async runtimes, Send/Sync bounds, async trait objects, etc. C++20, coroutines. Go, goroutines. Java21+, virtual threads

Note that C++ coroutines use heap allocation to avoid the problems that Pin is solving, which is a pretty big carve-out from the "zero overhead principle" that C++ usually aims for. The long development time of async traits has also been related to Rust not heap allocating futures. Whether that performance+portability-vs-complexity tradeoff is worth it for any given project is, of course, a different question.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#90

Post by no_wizard »

kibwen wrote: post #60
newpavlov wrote: post #49

Earlier quoted context omitted.

>which we found in every existing futures implementation we inspected This is exactly what I meant when I wrote about the indirect influence from other languages. People may dress it up as much as they want, but it's clear that polling was the most important model at the time (outside of the Windows world) and a lot of design consideration was put into being compatible with it. The Rust async model literally uses the…

> But the stackfull model is The existence of advantages doesn't change anything here. The problems is that the disadvantages made this approach a non-starter, despite a lot of effort to make it work. Tradeoffs exist in language design, and the approaches were judged accordingly. What works for Go doesn't necessarily work for Rust, because they target different domains. > I am really confident that if the async syste…

What were the original problems exactly? From what I recall they effectively boiled down to size concerns due to seeing themselves as a c/c++ successor and they didn’t want to lose any adoption in the embedded systems target audience.
Post reply on HN