Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Io_uring, kTLS and Rust for zero syscall HTTPS server

blog.habets.se

Post reply on HN
Print view View original

121–130 of 173 posts

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#121

Post by 0x457 »

duped wrote: post #62
newpavlov wrote: post #42

Earlier quoted context omitted.

This actually one of my many gripes about Rust async and why I consider it a bad addition to the language in the long term. The fundamental problem is that rust async was developed when epoll was dominant (and almost no one in the Rust circles cared about IOCP) and it has heavily influenced the async design (sometimes indirectly through other languages). Think about it for a second. Why do we not have this problem wi…

> You simply can not do it with state machines polled in user code That's not really true. The only guarantees in Rust futures are that they are polled() once and must have their Waker's wake() called before they are polled again. A completion based future submits the request on first poll and calls wake() on completion. That's kind of the interesting design of futures in Rust - they support polling and completion. T…

> And due to instability of a few features (async trait, return impl trait in trait, etc) there is not really a standard way to write executor independent async code (you can, some big crates do, but it's not necessarily trivial).

Uhm all of that is just sugar on top of stable feature. None of these features or lack off prevent portability.

Full portability isn't possible specifically due to how Waker works (i.e. is implementation specific). That allows async to work with different style of asyncs. Reason why io_uring is hard in rust is because of io_uring way of dealing with memory.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#122

Post by mgaunard »

thomashabets2 wrote: post #36
mgaunard wrote: post #22

"zero syscall" > In order to avoid busy looping, both the kernel and the web server will only busy-loop checking the queue for a little bit (configurable, but think milliseconds), and if there’s nothing new, the web server will do a syscall to “go to sleep” until something gets added to the queue.

Under load it's zero syscall (barring any rare allocations inside rustls for the handshake. I can't guarantee that it never does). Without load the overhead of calling (effectively) sleep() is, while technically true, not relevant. But sure, you can tweak the busyloop timers and burn 100% CPU on kernel and user side indefinitely if you want to avoid that sleep-when-idle syscall. It's just… not a good idea.

In my experience, trying to use io_uring for spinning/no-system-call uses is not straightforward.

First, there are some tricks required to actually make it work at all, then there is a problem that you'll need a core not only for userland, but also inside the kernel, both of them per-application.

Sharing a kernel spinning thread across multiple applications is also possible but requires further efforts (you need to share some parent ring across processes, which need to be related).

Overall I feel that it doesn't really deliver on the no-system-call idea, certainly not out of the box. You might have a more straightforward experience with XDP, which coincidentally gives you a lot more access and control as well if you need it.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#123

Post by danudey »

butterisgood wrote: post #78

Where do people get the idea that one thread per core is correct on a system that deals with time slices? In my experience “oversubscribing” threads to cores (more threads than cores) provides a wall-clock time benefit. I think one thread per core would work better without preemptive scheduling. But then we aren’t talking about Unix.

One thread per core if you're CPU-bound and not IO-bound.

In this very specific case, it seems as though the vast majority of the webserver's work is asynchronous and event-based, so the actual webserver is never waiting on I/O input or output - once it's ready you dump it somewhere the kernel can get to it and move on to the next request if there is one.

I think this gets this specific project close to the platonic ideal of a one-thread-per-core workload if indeed you're never waiting on I/O or any syscalls, but I feel as though it should come with extreme caveats of "this is almost never how the real world works so don't go artificially limiting your application to `nproc` threads without actually testing real-world use cases first".

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#124

Post by j_seigh »

api wrote: post #47

This is impressive but it’s also an amazing amount of complexity and difficult programming to work around the fact that syscalls are so slow. It seems like there’s these fundamental things in OSes that we just can’t improve, or I suppose can’t without breaking too much backward compatibility, so we are forced to do this.

I don't think it has to be. Conceptually it's just a couple of queues.

There's a software equivalent of the Peter Principle where software or an API becomes increasingly complex to the point where no one understands it. They then attempt to fix that by adding more functionality (complexity).

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#125

Post by hobofan »

duped wrote: post #62
newpavlov wrote: post #42

Earlier quoted context omitted.

This actually one of my many gripes about Rust async and why I consider it a bad addition to the language in the long term. The fundamental problem is that rust async was developed when epoll was dominant (and almost no one in the Rust circles cared about IOCP) and it has heavily influenced the async design (sometimes indirectly through other languages). Think about it for a second. Why do we not have this problem wi…

> You simply can not do it with state machines polled in user code That's not really true. The only guarantees in Rust futures are that they are polled() once and must have their Waker's wake() called before they are polled again. A completion based future submits the request on first poll and calls wake() on completion. That's kind of the interesting design of futures in Rust - they support polling and completion. T…

> The only guarantees in Rust futures are that they are polled() once and must have their Waker's wake() called before they are polled again.

I just had to double-check as this sounded strange to me, and no that's not true.

The most efficient design is to do it that way, yes, but there are no guarantees of that sort. If one wants to build a less efficient executor, it's perfectly permissible to just poll futures on a tight loop without involving the Waker at all.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#126

Post by hobofan »

npalli wrote: post #63

So, current status on async Rust - you need to understand: Futures, Pin, Waker, async runtimes, Send/Sync bounds, async trait objects, etc. C++20, coroutines. Go, goroutines. Java21+, virtual threads

If you are fine with writing "good enough" high-level Rust code (that will potentially still beat out most other languages in terms of performance) and are fine with using the mid-level primitives that other people have built, you don't really have to understand most of those things.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#127

Post by wahern »

butterisgood wrote: post #78

Where do people get the idea that one thread per core is correct on a system that deals with time slices? In my experience “oversubscribing” threads to cores (more threads than cores) provides a wall-clock time benefit. I think one thread per core would work better without preemptive scheduling. But then we aren’t talking about Unix.

In the case of io_uring, one user thread per core is not a bad rule of thumb given that the kernel side is using a pool of worker threads.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#129

Post by duped »

hobofan wrote: post #125
duped wrote: post #62

Earlier quoted context omitted.

> You simply can not do it with state machines polled in user code That's not really true. The only guarantees in Rust futures are that they are polled() once and must have their Waker's wake() called before they are polled again. A completion based future submits the request on first poll and calls wake() on completion. That's kind of the interesting design of futures in Rust - they support polling and completion. T…

> The only guarantees in Rust futures are that they are polled() once and must have their Waker's wake() called before they are polled again. I just had to double-check as this sounded strange to me, and no that's not true. The most efficient design is to do it that way, yes, but there are no guarantees of that sort. If one wants to build a less efficient executor, it's perfectly permissible to just poll futures on a…

Let me rephrase, there's no guarantee that a poll() is called again (because of cancel safety) and in practice you have to call wake() because executors won't reschedule the task unless one of their children wake()s

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#130

Post by WJW »

Pretty cool! Adding kTLS is definitely an improvement. I made an actually zero-syscall per request server a few years ago (and blogged about it at https://wjwh.eu/posts/2021-10-01-no-syscall-server-iouring.h...) but as TFA notes it comes at a heavy cost of constantly busy-looping.

io_uring is very cool tech though and has been progressing at an impressive pace the last few years.

Post reply on HN