Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Io_uring, kTLS and Rust for zero syscall HTTPS server

blog.habets.se

Post reply on HN
Print view View original

51–60 of 173 posts

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#51

Post by johncolanduoni »

Seattle3503 wrote: post #9

> For example when submitting a write operation, the memory location of those bytes must not be deallocated or overwritten. > The io-uring crate doesn’t help much with this. The API doesn’t allow the borrow checker to protect you at compile time, and I don’t see it doing any runtime checks either. I've seen comments like this before[1], and I get the impression that building a a safe async Rust library around io_urin…

It’s annoying but possible to do this correctly and not have the API be too bad. The “happy path” of a clean success or error is fine if you accept that buffers can’t just be simple &[u8] slices. Cancellation can be handled safely with something like the following API contract:

Have your function signature be async fn read(buffer: &mut Vec) -> Result’ (you can use something more convenient like ‘&mut BytesMut’ too). If you run the future to completion (success or failure), the argument holds the same buffer passed in, with data filled in appropriately on success. If you cancel/drop the future, the buffer may point at an empty allocation instead (this is usually not an annoying constraint for most IO flows, and footgun potential is low).

The way this works is that your library “takes” the underlying allocation before starting the operation out of the variable, replacing it with the default unallocated ‘Vec’. Once the buffer is no longer used by the IO system, it puts it back before returning. If you cancel, it manages the buffer in the background to release it when safe and the unallocated buffer is left in the passed variable.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#52

Post by johncolanduoni »

aliceryhl wrote: post #50
Seattle3503 wrote: post #9

> For example when submitting a write operation, the memory location of those bytes must not be deallocated or overwritten. > The io-uring crate doesn’t help much with this. The API doesn’t allow the borrow checker to protect you at compile time, and I don’t see it doing any runtime checks either. I've seen comments like this before[1], and I get the impression that building a a safe async Rust library around io_urin…

> IIRC Alice from the tokio team also suggested there hasn't been much interest in pushing through these difficulties more recently, as the current performance is "good enough". Well, I think there is interest, but mostly for file IO. For file IO, the situation is pretty simple. We already have to implement that using spawn_blocking, and spawn_blocking has the exact same buffer challenges as io_uring does, so transla…

This covers probably 90% of the usefulness of io_uring for non-niche applications. Its original purpose was doing buffered async file IO without a bunch of caveats that make it effectively useless. The biggest speed up I’ve found with it is ‘stat’ing large sets of files in the VFS cache. It can literally be 50x faster at that, since you can do 1000 files with a single systemcall and the data you need from the kernel is all in memory.

High throughput network usecases that don’t need/want AF_XDP or DPDK can get most of the speedup with ‘sendmmsg/recvmmsg’ and segmentation offload.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#53

Post by selfmodruntime »

LAC-Tech wrote: post #33

I think rusts glacial compile times prevent it from being a useful platform for web apps. Yes it's a nice language, and very performant, but it's horrible devex to have to wait seconds for your server to recompile after a change.

Compile times aren’t glacial and will be much faster with the new trait solver and cranelift.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#55

Post by johncolanduoni »

Tuna-Fish wrote: post #28
JoshTriplett wrote: post #10

Earlier quoted context omitted.

I think the right way to build a safe interface around io_uring would be to use ring-owned buffers, ask the ring for a buffer when you want one, and give the buffer back to the ring when initiating a write.

This works perfectly well, and allows using the type system to handle safety. But it also really limits how you handle memory, and makes it impossible to do things like filling out parts of existing objects, so a lot of people are reluctant to take the plunge.

That’s annoying for people writing bespoke low-level networking code, but for a high-level HTTP library it’s a rounding error in the overall complexity on display. I think the bigger barrier for Tokio is that the interplay between having an epoll instance and a io_uring instance on the same pool is problematic and can erase performance gains. If done greenfield you could implement the “normal” APIs with ‘IORING_OP_POLL_ADD’, but not all of the exposed ‘mio’ surface area can work this way - only the oneshot API.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#56

Post by drewg123 »

phrotoma wrote: post #38

Anybody know what the state of kTLS is? I asked one of the Cilium devs about it a while ago'cause I'd seen Thomas Graf excitedly talking about it and he told me that kernel support in many distros was lacking so they aren't ready to enable it by default.

That's a shame. How hard is it to enable? Do you need a custom kernel, or can you enable it at runtime?

On FreeBSD, its been in the kernel / openssl since 13, and has been one runtime toggle (sysctl kern.ipc.tls.enable=1) away from being enabled. And its enabled by default in the upcoming FreeBSD-15.

We (at Netflix) have run all of our tls encrypted streaming over kTLS for most of a decade.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#57

Post by andyferris »

johncolanduoni wrote: post #51
Seattle3503 wrote: post #9

> For example when submitting a write operation, the memory location of those bytes must not be deallocated or overwritten. > The io-uring crate doesn’t help much with this. The API doesn’t allow the borrow checker to protect you at compile time, and I don’t see it doing any runtime checks either. I've seen comments like this before[1], and I get the impression that building a a safe async Rust library around io_urin…

It’s annoying but possible to do this correctly and not have the API be too bad. The “happy path” of a clean success or error is fine if you accept that buffers can’t just be simple &[u8] slices. Cancellation can be handled safely with something like the following API contract: Have your function signature be async fn read(buffer: &mut Vec ) -> Result ’ (you can use something more convenient like ‘&mut BytesMut’ too)…

It sounds like this would be better modelled by passing ownership of the buffer and expecting it to be returned on the success (ok) case. What you described doesn't seem compatible with what I would call a mutable borrow (mutate the contents of a Vec).

Or maybe I've misunderstood?

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#58

Post by andyferris »

Soft wrote: post #45
IshKebab wrote: post #35

Earlier quoted context omitted.

How is that different to Fn(_: &mut T) ?

In the former the caller does not retain access to T until Fn returns.

I think I'm lost. If I give a mutable reference to a function... I can't access it (even read it) until it returns, no?

What is different?

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#60

Post by kibwen »

newpavlov wrote: post #49
kibwen wrote: post #46

Earlier quoted context omitted.

> The fundamental problem is that rust async was developed when epoll was dominant (and almost no one in the Rust circles cared about IOCP) No, this is a mistaken retelling of history. The Rust developers were not ignorant of IOCP, nor were they zealous about any specific async model. They went looking for a model that fit with Rust's ethos, and completion didn't fit. Aaron Turon has an illuminating post from 2016 ex…

>which we found in every existing futures implementation we inspected This is exactly what I meant when I wrote about the indirect influence from other languages. People may dress it up as much as they want, but it's clear that polling was the most important model at the time (outside of the Windows world) and a lot of design consideration was put into being compatible with it. The Rust async model literally uses the…

> But the stackfull model is

The existence of advantages doesn't change anything here. The problems is that the disadvantages made this approach a non-starter, despite a lot of effort to make it work. Tradeoffs exist in language design, and the approaches were judged accordingly. What works for Go doesn't necessarily work for Rust, because they target different domains.

> I am really confident that if the async system was designed today we would've had a totally different model

No, without solving the original problems, the outcome would be the same. The Rust devs at the time were well aware of io_uring.

Post reply on HN