Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Io_uring, kTLS and Rust for zero syscall HTTPS server

blog.habets.se

Post reply on HN
Print view View original

131–140 of 173 posts

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#131

Post by commandersaki »

bmcahren wrote: post #6

This was a good read and great work. Can't wait to see the performance tests. Your write up connected some early knowledge from when I was 11 where I was trying to set up a database/backend and was finding lots of cgi-bin online. I realize now those were spinning up new processes with each request https://en.wikipedia.org/wiki/Common_Gateway_Interface I remember when sendfile became available for my large gaming foru…

I'm sceptical of the efficiency gains with sendfile; seems marginal at best, even in the late 90s when it was at the height of popularity.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#132

Post by thinkharderdev »

butterisgood wrote: post #103
gorset wrote: post #80

Earlier quoted context omitted.

Isolating a core and then pinning a single thread is the way to go to get both low latency and high throughput, sacrificing efficiency. This works fine on Linux, and common approach for trading systems where it’s fine to oversubscribe a bunch of cores for this type of stuff. The cores are mostly busy spinning and doing nothing, so it’s very inefficient in terms of actual work, but great for latency and throughput whe…

I just wish people who give this advice for 1 thread per core would "expand their reasoning" or "show the work". It's not blanket good advice for all things.

It is definitely not good advice for all things. For workloads that are either end of the CPU/IO spectrum (e.g. almost all waiting on IO or almost all doing CPU work) it can be a huge win as you can get very good L1 cache utilization, are not context-switching and don't need to handle thread synchronization in your code because not state is shared between threads.

For workloads that are a mix of IO and non-trivial CPU work, it can still work but is much, much harder to get right.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#133

Post by Asmod4n »

vlovich123 wrote: post #120
newpavlov wrote: post #88

Earlier quoted context omitted.

>is that a failure of the async model This, 100%. Being really generous, it can be called a leaky model which is poorly compatible with completion-based APIs.

The leaky model is that you could ever receive into a stack buffer and you're arguing to persist this model. The reason it's leaky is that copying memory around is supremely expensive. But that's how the BSD socket API from the 90s works and btw something you can make work with async provided you're into memory copies. io_uring is a modern API that's for performance and that's why Rust libraries try to avoid memory c…

there is just one catch.

Using the feature to let io_uring handle buffers for you limits you to the mem lock limit of a process, which is 8MB on a typical debian install (more on others) And that's a hard limit unless you got root access to said machine.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#134

Post by vlovich123 »

Asmod4n wrote: post #133
vlovich123 wrote: post #120

Earlier quoted context omitted.

The leaky model is that you could ever receive into a stack buffer and you're arguing to persist this model. The reason it's leaky is that copying memory around is supremely expensive. But that's how the BSD socket API from the 90s works and btw something you can make work with async provided you're into memory copies. io_uring is a modern API that's for performance and that's why Rust libraries try to avoid memory c…

there is just one catch. Using the feature to let io_uring handle buffers for you limits you to the mem lock limit of a process, which is 8MB on a typical debian install (more on others) And that's a hard limit unless you got root access to said machine.

Sure, that's the most efficient way. But you can still have the user allocate a read buffer, pass it to the read API & receive it on the way out. In fact, unlike what OP claimed, this is actually more efficient since you could safely avoid unnecessarily initializing this buffer safely (by truncating to the length read before returning) whereas safely using uninitialized buffers is kind of tricky.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#135

Post by kev009 »

jabl wrote: post #43
kev009 wrote: post #12

Earlier quoted context omitted.

It wasn't just CGI, every HTTP session was commonly a forked copy of the entire server in the CERN and Apache lineage! Apache gradually had better answers, but their API with common addons made it a bit difficult to transition so webservers like nginx took off which are built closer to the architecture in the article with event driven I/O from the beginning.

To nitpick at least as of Apache HTTPD 1.3 ages ago it wasn't forking for every request, but had a pool of already forked worker processes with each handling one connection at a time but could handle an unlimited number of connections sequentially, and it could spawn or kill worker processes depending on load. The same model is possible in Apache httpd 2.x with the "prefork" mpm.

I don't see anything in my comment that implied _when_ the forking happened so it's not really a nit :)

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#136

Post by lossolo »

commandersaki wrote: post #131
bmcahren wrote: post #6

This was a good read and great work. Can't wait to see the performance tests. Your write up connected some early knowledge from when I was 11 where I was trying to set up a database/backend and was finding lots of cgi-bin online. I realize now those were spinning up new processes with each request https://en.wikipedia.org/wiki/Common_Gateway_Interface I remember when sendfile became available for my large gaming foru…

I'm sceptical of the efficiency gains with sendfile; seems marginal at best, even in the late 90s when it was at the height of popularity.

> seems marginal at best

Depends on the workload.

Normally you would go read() -> write() so:

1. Disk -> page cache (DMA)

2. Kernel -> user copy (read)

3. User -> kernel copy (write)

4. Kernel -> NIC (DMA)

sendfile():

1. Disk -> page cache (DMA)

No user space copies, kernel wires those pages straight to the socket

2. Kernel -> NIC (DMA)

So basically, it eliminates 1-2 memory copies along with the associated cache pollution and memory bandwidth overhead. If you are running high QPS web services where syscall and copy overheads dominate, for example CDNs/static file serving the gains can be really big. Based on my observations this can mean double digit reductions in CPU usage and up to ~2x higher throughput.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#137

Post by kev009 »

commandersaki wrote: post #131
bmcahren wrote: post #6

This was a good read and great work. Can't wait to see the performance tests. Your write up connected some early knowledge from when I was 11 where I was trying to set up a database/backend and was finding lots of cgi-bin online. I realize now those were spinning up new processes with each request https://en.wikipedia.org/wiki/Common_Gateway_Interface I remember when sendfile became available for my large gaming foru…

I'm sceptical of the efficiency gains with sendfile; seems marginal at best, even in the late 90s when it was at the height of popularity.

Then you don't understand the memory and protection model of a modern system very well.

sendfile effectively turns your user space file server into a control plane, and moves the data plane to where the data is eliminating copies between address spaces. This can be made congruent with I/O completions (i.e. Ethernet+IP and block) and made asynchronous so the entire thing is pumping data between completion events. Watch the Netflix video the author links in the post.

There is an inverted approach where you move all this into a single user address space, i.e. DPDK, but it's the same overall concept just a different who.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#138

Post by mpyne »

oconnor663 wrote: post #89
npalli wrote: post #63

So, current status on async Rust - you need to understand: Futures, Pin, Waker, async runtimes, Send/Sync bounds, async trait objects, etc. C++20, coroutines. Go, goroutines. Java21+, virtual threads

Note that C++ coroutines use heap allocation to avoid the problems that Pin is solving, which is a pretty big carve-out from the "zero overhead principle" that C++ usually aims for. The long development time of async traits has also been related to Rust not heap allocating futures. Whether that performance+portability-vs-complexity tradeoff is worth it for any given project is, of course, a different question.

C++ coroutines must allocate at runtime as the allocation size isn't resolvable early enough at compile time to statically fix the allocation, but it's not required to be allocated from the heap (not that custom allocators are fun, but it is possible).

In any event it's essentially a stack frame so it's not a failure of zero-overhead, the stack frame will need to be somewhere.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#139

Post by touisteur »

Seattle3503 wrote: post #9

> For example when submitting a write operation, the memory location of those bytes must not be deallocated or overwritten. > The io-uring crate doesn’t help much with this. The API doesn’t allow the borrow checker to protect you at compile time, and I don’t see it doing any runtime checks either. I've seen comments like this before[1], and I get the impression that building a a safe async Rust library around io_urin…

I wish I could have been paid to work on SPARK specification around io_uring so that one could have built on it. Or to work on SPARK-to-eBPF (there's already a llvm backend for gnat) and have some form of guarantees at the seams... alas.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#140

Post by butterisgood »

danudey wrote: post #123
butterisgood wrote: post #78

Where do people get the idea that one thread per core is correct on a system that deals with time slices? In my experience “oversubscribing” threads to cores (more threads than cores) provides a wall-clock time benefit. I think one thread per core would work better without preemptive scheduling. But then we aren’t talking about Unix.

One thread per core if you're CPU-bound and not IO-bound. In this very specific case, it seems as though the vast majority of the webserver's work is asynchronous and event-based, so the actual webserver is never waiting on I/O input or output - once it's ready you dump it somewhere the kernel can get to it and move on to the next request if there is one. I think this gets this specific project close to the platonic…

But, your CPU availability is time sliced... So, why is not "more than one thread per core" equivalent to "more CPU" (my point is, sometimes it is...)
Post reply on HN