Io_uring, kTLS and Rust for zero syscall HTTPS server • Hacker News Board

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#151

Post by commandersaki » Sat, Aug 23, 2025, 12:02 AM UTC

lossolo wrote: post #136

commandersaki wrote: post #131

Earlier quoted context omitted.

I'm sceptical of the efficiency gains with sendfile; seems marginal at best, even in the late 90s when it was at the height of popularity.

> seems marginal at best Depends on the workload. Normally you would go read() -> write() so: 1. Disk -> page cache (DMA) 2. Kernel -> user copy (read) 3. User -> kernel copy (write) 4. Kernel -> NIC (DMA) sendfile(): 1. Disk -> page cache (DMA) No user space copies, kernel wires those pages straight to the socket 2. Kernel -> NIC (DMA) So basically, it eliminates 1-2 memory copies along with the associated cache pol…

I understand the optimisation, I'm just saying I'm sceptical the optimisation is even that useful, like it seems it'd only kick in with pathological cases where kernel round trip time is really dominating; my gut reckons most applications just don't benefit. Caddy in the last few years got sendfile support and with it on and off and it usually you wouldn't see a discernible difference [1].

Which makes me sceptical for the argument for kTLS which is stated in the article; what benefit does offloading your crypto to the kernel provider (possibly making it more brittle). I've seen the author of haproxy say that performance he's seen has been only marginal, but did point out it was useful in that you can strace your process and see plaintext instead of ciphertext which is nice.

[1]: https://blog.tjll.net/reverse-proxy-hot-dog-eating-contest-c...

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#152

Post by mastax » Sat, Aug 23, 2025, 1:03 AM UTC

dpeckett wrote: post #76

johncolanduoni wrote: post #52

Earlier quoted context omitted.

This covers probably 90% of the usefulness of io_uring for non-niche applications. Its original purpose was doing buffered async file IO without a bunch of caveats that make it effectively useless. The biggest speed up I’ve found with it is ‘stat’ing large sets of files in the VFS cache. It can literally be 50x faster at that, since you can do 1000 files with a single systemcall and the data you need from the kernel…

For TCP streams syscall overhead isn't a big issue really, you can easily transfer large chunks of data in each write(). If you have TCP segmentation offload available you'll have no serious issues pushing 100gbit/s. Also if you are sending static content don't forget sendfile(). UDP is a whole another kettle of fish, get's very complicated to go above 10gbit/s or so. This is a big part of why QUIC really struggles t…

I have a hard time believing that google is serving YouTube over QUIC/HTTP3 at 10Gbit/s, or even 30Gbit/s.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#153

Post by fpoling » Sat, Aug 23, 2025, 1:44 AM UTC

kibwen wrote: post #46

newpavlov wrote: post #42

Earlier quoted context omitted.

This actually one of my many gripes about Rust async and why I consider it a bad addition to the language in the long term. The fundamental problem is that rust async was developed when epoll was dominant (and almost no one in the Rust circles cared about IOCP) and it has heavily influenced the async design (sometimes indirectly through other languages). Think about it for a second. Why do we not have this problem wi…

> The fundamental problem is that rust async was developed when epoll was dominant (and almost no one in the Rust circles cared about IOCP) No, this is a mistaken retelling of history. The Rust developers were not ignorant of IOCP, nor were they zealous about any specific async model. They went looking for a model that fit with Rust's ethos, and completion didn't fit. Aaron Turon has an illuminating post from 2016 ex…

That post explicitly stated one of the goals was to avoid requiring heap allocations. But fundamentally io_uring is incompatible with the stack and in practice coding against it requires dynamic allocations. If that would be known 10 years ago, surely it would have influenced the design goals.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#154

Post by johncolanduoni » Sat, Aug 23, 2025, 2:29 AM UTC

mastax wrote: post #152

dpeckett wrote: post #76

Earlier quoted context omitted.

For TCP streams syscall overhead isn't a big issue really, you can easily transfer large chunks of data in each write(). If you have TCP segmentation offload available you'll have no serious issues pushing 100gbit/s. Also if you are sending static content don't forget sendfile(). UDP is a whole another kettle of fish, get's very complicated to go above 10gbit/s or so. This is a big part of why QUIC really struggles t…

I have a hard time believing that google is serving YouTube over QUIC/HTTP3 at 10Gbit/s, or even 30Gbit/s.

These are per-connection bottlenecks, largely due to implementation choices in the Linux network stack. Even with vanilla Linux networking, vertical scale can get the aggregate bandwidth as high as you want if you don’t need 10G per connection (which YouTube doesn’t), as long as you have enough CPU cores and NIC queues.

Another thing to consider: Google’s load balancers are all bespoke SDN and they almost certainly speak HTTP1/2 between the load balancers and the application servers. So Linux network stack constraints are probably not relevant for the YouTube frontend serving HTTP3 at all.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#155

Post by fpoling » Sat, Aug 23, 2025, 3:26 AM UTC

no_wizard wrote: post #102

const_cast wrote: post #95

Earlier quoted context omitted.

I mean from an outsiders perspective on Rust this is how I saw it. Rust is in a strange place because they're a systems language directly competing with C++. Async, in general, doesn't vibe with that but green threads definitely don't. If you're gonna do green threads you might as well throw in a GC too and get a whole runtime. And now you're writing Go.

I don't think doing green threads equates to 'well might as well have a GC now!'. I think they made the wrong tradeoff too, because hardware will inevitably catch up to the language requirements, especially if its desirable to use. Not to mention over time things can be made more efficient from the Rust side as well, with compiler improvements, better programming techniques etc. I think they made the wrong bet, perso…

Hardware does not catches up with language requirements. If anything, it is languages/compilers that catch up with hardware, like SSE instructions and loop parallel ism.

For me the mistake that Rust made was that it tried too hard to behave like C/C++ with its single execution stack.

Ada uses two stacks allowing a callee to return a stack-allocated arrays to the caller. Not only it allows to avoid dynamic allocations in many cases where C++ allocates memory, but it also reduces the need for pointers making the code safer even without the borrow checker.

If instead of async Rust spent efforts on implementing something like that or even allow for explicit stack control from safe code so green threads or co-routines could be implemented as a library it could be more compatible with io_uring world.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#156

Post by Inufu » Sat, Aug 23, 2025, 5:34 AM UTC

vlovich123 wrote: post #118

Inufu wrote: post #114

Earlier quoted context omitted.

Requiring ownership transfer gives up on one of the main selling points of Rust, being able to verify reference lifetime and safety at compile time. If we have to give up on references then a lot of Rusts complexity no longer buys us anything.

I'm not sure what you're trying to say, but the compile-time safety requirement isn't given up. It would look something like: self.buffer = io_read(self.buffer)? This isn't much different than io_read(&mut self.buffer)? since rust doesn't permit simultaneous access when a mutable reference is taken.

It means you can for example no longer do things like get multiple disjoint references into the same buffer for parallel reads/writes of independent chunks.

Or well you can, using unsafe, Arc and Mutex - but at that point the safety guarantees aren’t much better than what I get in well designed C++.

Don’t get me wrong, I still much prefer Rust, but I wish async and references worked together better.

Source: I recently wrote a high-throughput RPC library in Rust (saturating > 100 Gbit NICs)

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#157

Post by zozbot234 » Sat, Aug 23, 2025, 5:57 AM UTC

fpoling wrote: post #155

no_wizard wrote: post #102

Earlier quoted context omitted.

I don't think doing green threads equates to 'well might as well have a GC now!'. I think they made the wrong tradeoff too, because hardware will inevitably catch up to the language requirements, especially if its desirable to use. Not to mention over time things can be made more efficient from the Rust side as well, with compiler improvements, better programming techniques etc. I think they made the wrong bet, perso…

Hardware does not catches up with language requirements. If anything, it is languages/compilers that catch up with hardware, like SSE instructions and loop parallel ism. For me the mistake that Rust made was that it tried too hard to behave like C/C++ with its single execution stack. Ada uses two stacks allowing a callee to return a stack-allocated arrays to the caller. Not only it allows to avoid dynamic allocations…

> Ada uses two stacks allowing a callee to return a stack-allocated arrays to the caller.

You could do this manually by threading a pointer to a separately-allocated stack (could be on the heap or perhaps just a static allocation) as an extra function parameter. It's just a very simple case of arena allocation, with similar advantages and disadvantages. (For example, the caller must ensure that enough space is available on the dynamic-data stack for anything that the callee might want to push onto it.) In general it's just not really worth it, because it turns out that dynamically-sized data that one would not want to simply place on the heap is rare anyway.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#158

Post by LAC-Tech » Sat, Aug 23, 2025, 6:47 AM UTC

craftkiller wrote: post #107

maeln wrote: post #37

Earlier quoted context omitted.

> but it's horrible devex to have to wait seconds for your server to recompile after a change. What a time to be alived that seconds to recompile is consider horrible devex.

At my first job out of college it took 30 minutes to recompile and launch the server. Now the kids complain about 10 seconds. It's just impossible for me to take their complaints seriously. 10 seconds isn't even enough time for a mental context-switch, its just slightly more time than "instant". Back in the day, something like this wasn't an exaggeration: https://xkcd.com/303/

I can remember instant reloads of application servers on a job 10+ years ago grandpa. This isn't new.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#159

Post by kev009 » Sat, Aug 23, 2025, 7:02 AM UTC

tliltocatl wrote: post #71

kev009 wrote: post #12

Earlier quoted context omitted.

It wasn't just CGI, every HTTP session was commonly a forked copy of the entire server in the CERN and Apache lineage! Apache gradually had better answers, but their API with common addons made it a bit difficult to transition so webservers like nginx took off which are built closer to the architecture in the article with event driven I/O from the beginning.

That's because Unix API used to assume fork() is extremely cheap. Threads were ugly performance hack second-class citizens - still are in some ways. This was indeed true on PDP-11 (just copy a <64KB disk file!), but as address spaces grew, it became prohibitively expensive to copy page tables, so programmers turned to multithreading. At then multicore CPUs became the norm, and multithreading on multicore CPUs meant a…

It's also a pretty bold scheduler benchmark to be handling tens of thousands of processes or 1:1 thread wakeups, especially the further back in time you go considering fairness issues. And then that's running at the wrong latency granularity for fast I/O completion events across that many nodes so it's going to run like a screen door on a submarine without a lot of rethinking things.

Evented I/O works out pretty well in practice for the I and D cache, especially if you can affine and allocate things as the article states, and do similar natural alignments inside the kernel (i.e. RSS/consistent hashing).

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#160

Post by alfiedotwtf » Sat, Aug 23, 2025, 2:12 PM UTC

kbolino wrote: post #96

alfiedotwtf wrote: post #48

Earlier quoted context omitted.

In my universe, `let` wouldn’t exist… instead there would only be 3 ways to declare variables: 1. global my_global_var: GlobalType = … 2. heap my_heap_var: HeapType = … 3. stack my_stack_var: StackType = … Global types would need to implement a global trait to ensure mutual exclusion (waves hands). So by having the location of allocation in the type itself, we no longer have to do boxing mental gymnastics

But what does "heap my_heap_var" actually mean, without a garbage collector? Who owns "my_heap_var" and when does it get deallocated? What does explicitly writing out the heap-ness of a variable ultimately provide, that Rust's existing type system with its many heap-allocated types (Box, Rc, Arc, Vec, HashMap, etc.) doesn't already provide?

> What does explicitly writing out the heap-ness of a variable ultimately provide, that Rust's existing type system with its many heap-allocated types (Box, Rc, Arc, Vec, HashMap, etc.) doesn't already provide?

To be honest, I was thinking more in terms of cognitive overload i.e. is all that Box boilerplate even needed if we were to treat all `heap my_heap = …” as box underneath? In other words, couldn’t we elide all that away:

    let foo = Box::new(MyFoo::default ());

Becomes:

    heap foo = MyFoo::default();

Must nicer!