Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Io_uring, kTLS and Rust for zero syscall HTTPS server

blog.habets.se

Post reply on HN
Print view View original

161–170 of 173 posts

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#161

Post by zbentley »

vlovich123 wrote: post #120
newpavlov wrote: post #88

Earlier quoted context omitted.

>is that a failure of the async model This, 100%. Being really generous, it can be called a leaky model which is poorly compatible with completion-based APIs.

The leaky model is that you could ever receive into a stack buffer and you're arguing to persist this model. The reason it's leaky is that copying memory around is supremely expensive. But that's how the BSD socket API from the 90s works and btw something you can make work with async provided you're into memory copies. io_uring is a modern API that's for performance and that's why Rust libraries try to avoid memory c…

> what you really want is to ask io_uring to allocate the pages itself so that for reads it gives you pages that were allocated by the kernel

Okay, but what about writes? If I have a memory region that I want io_uring to write, it's a major pain in the ass to manage the lifetime of objects in that region in a safe way. My choices are basically: manually manage the lifetime and only allow it to be dropped when I see a completion show up (this is what most everything does now, and it's a) hard to get right and b) limited in many ways, e.g. it's heap-only), or permanently leak that memory as unusable.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#162

Post by zbentley »

andyferris wrote: post #58
Soft wrote: post #45

Earlier quoted context omitted.

In the former the caller does not retain access to T until Fn returns.

I think I'm lost. If I give a mutable reference to a function... I can't access it (even read it) until it returns, no? What is different?

Let's say a function "foo" calls "fn bar(_: &mut T) -> ()".

When passing a mutable reference, the lifetime of the object is largely decided by "foo" (with some caveats).

Now, let's say that "foo" instead calls "fn bar(_: T) -> T".

When passing the object itself, the lifetime is largely decided/decide-able by "bar".

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#163

Post by zbentley »

tsimionescu wrote: post #30
avar wrote: post #26

Earlier quoted context omitted.

every HTTP session was commonly a forked copy of the entire server in the CERN and Apache lineage! And there's nothing wrong with that for application workers. On *nix systems fork() is very fast, you can fork "the entire server" and the kernel will only COW your memory. As nginx etc. showed you can get better raw file serving performance with other models, but it's still a legitimate technique for application logic…

Forking for anything other than calling exec is still a horrible idea (with special exceptions like shells). Forking is a very unsafe operation (you can easily share locks and files with the child process unless both your code and every library you use is very careful - for example, it's easy to get into malloc deadlocks with forked processes), and its performance depends a lot on how you actually use it.

I think it's not quite that bad (and I know that this has been litigated to death all over the programmer internet).

If you are forking from a language/ecosystem that is extremely thread-friendly, (e.g. Go, Java, Erlang) fork is more risky. This is because such runtimes mean a high likelihood of there being threads doing fork-unsafe things at the moment of fork().

If you are forking from a language/ecosystem that is thread-unfriendly, fork is less risky. That isn't to say "it's always safe/low risk to run fork() in e.g. Python, Ruby, Perl", but in those contexts it's easier to prove/test invariants like "there are no threads running/so-and-so lock is not held at the point in my program when I fork", at which point the risks of fork(2) are much reduced.

To be clear, "reduced" is not the same as "gone"! You still have to reason about explicitly taken locks in the forking thread, file descriptors, signal handlers, and unexpected memory growth due to CoW/GC interactions. But that's a lot more tractable than the Java situation of "it's tricky to predict how many Java threads are active when I want to fork, and even trickier to know if there are any JNI/FFI-library-created raw pthreads running, the GC might be threaded, and checking for each of those things is still racy with my call to fork(2)".

You still have to make sure that that fork-safety invariants are true. But the effort to do that is extremely different depending on language platform.

Rust/C/C++ don't cleanly fit into either of those two (already mushy/subjective) categorizations, though. Whether forking is feasible in a given Rust/C/C++ codebase depends on what the code does and requires a tricky set of judgement calls and at-a-distance knowledge going forward to make sure that the codebase doesn't become fork-unsafe in harmful ways.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#164

Post by zbentley »

gpderetta wrote: post #146
ori_b wrote: post #77

Earlier quoted context omitted.

> Go: goroutines are not async Sure they are. The abstraction they provide is a synchronous API, but it's accomplished using an async runtime.

By that definition, pthread is also async. If everything is async, then the word loses all meanings. Async is really about the surface syntax and ergonomics, not the implementation.

Eh, not really. Async (in this semantic context) is generally about cooperative concurrency and also often about concurrent or multiplexed I/O. Pthreads aren't async by those definitions, though you can run async code within a given pthread as usual.

Goroutines are an unusual case, in that they don't have cooperative concurrency--they're pre-emptive--but the Go runtime does perform I/O using concurrent multiplexers under the hood.

So goroutines are kind of both: computation execution and code semantics look like pthreads, but I/O operations look like NodeJS on the backend.

Now, I'm not sure what "async runtime" means in the GP. If they're referring to I/O multiplexers, then they should say that. If they're referring to something else, then I'm not familiar with other uses of that term that would accurately apply to Golang.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#166

Post by ori_b »

zbentley wrote: post #164
gpderetta wrote: post #146

Earlier quoted context omitted.

By that definition, pthread is also async. If everything is async, then the word loses all meanings. Async is really about the surface syntax and ergonomics, not the implementation.

Eh, not really. Async (in this semantic context) is generally about cooperative concurrency and also often about concurrent or multiplexed I/O . Pthreads aren't async by those definitions, though you can run async code within a given pthread as usual. Goroutines are an unusual case, in that they don't have cooperative concurrency--they're pre-emptive--but the Go runtime does perform I/O using concurrent multiplexers…

Well, that's exactly what the kernel is doing when it swaps threads. When you block on I/O, you're voluntarily pausing your thread and doing concurrent I/O with another thread.

Async and threads are a lot closer than most people think. An OS is mainly a queue for swapping between async operations, and a collection of abstracted services that the async operations can request, like network or disk i/o.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#167

Post by alpb »

superb_dev wrote: post #128
alde wrote: post #75

Unfortunately io_uring is disabled by default on most cloud workload orchestrators, like CloudRun, GKE, EKS and even local Docker. Hope this will change soon, but until then it will remain very niche.

Why do they disable io_uring?

Security reasons. https://news.ycombinator.com/item?id=44632240 There are also other edge cases around cgroups accounting that renders some isolation/throttling mechanisms not fully effective.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#168

Post by vlovich123 »

zbentley wrote: post #161
vlovich123 wrote: post #120

Earlier quoted context omitted.

The leaky model is that you could ever receive into a stack buffer and you're arguing to persist this model. The reason it's leaky is that copying memory around is supremely expensive. But that's how the BSD socket API from the 90s works and btw something you can make work with async provided you're into memory copies. io_uring is a modern API that's for performance and that's why Rust libraries try to avoid memory c…

> what you really want is to ask io_uring to allocate the pages itself so that for reads it gives you pages that were allocated by the kernel Okay, but what about writes? If I have a memory region that I want io_uring to write, it's a major pain in the ass to manage the lifetime of objects in that region in a safe way. My choices are basically: manually manage the lifetime and only allow it to be dropped when I see a…

You ask the I/O system for a writable buffer. When you fill it up, you hand it off. Once the I/o finishes, it goes back into the available pool of memory to write with. This is how high performance I/O works.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#169

Post by craftkiller »

LAC-Tech wrote: post #158
craftkiller wrote: post #107

Earlier quoted context omitted.

At my first job out of college it took 30 minutes to recompile and launch the server. Now the kids complain about 10 seconds. It's just impossible for me to take their complaints seriously. 10 seconds isn't even enough time for a mental context-switch, its just slightly more time than "instant". Back in the day, something like this wasn't an exaggeration: https://xkcd.com/303/

I can remember instant reloads of application servers on a job 10+ years ago grandpa. This isn't new.

Yeah, we had hot reloading of code too but hot-reloading for instant "reloads" was needed back then. Nowadays, you can do a full relaunch of the server in 10 seconds so hot reloads no longer matter.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#170

Post by [deleted] »

ori_b wrote: post #166
zbentley wrote: post #164

Earlier quoted context omitted.

Eh, not really. Async (in this semantic context) is generally about cooperative concurrency and also often about concurrent or multiplexed I/O . Pthreads aren't async by those definitions, though you can run async code within a given pthread as usual. Goroutines are an unusual case, in that they don't have cooperative concurrency--they're pre-emptive--but the Go runtime does perform I/O using concurrent multiplexers…

Well, that's exactly what the kernel is doing when it swaps threads. When you block on I/O, you're voluntarily pausing your thread and doing concurrent I/O with another thread. Async and threads are a lot closer than most people think. An OS is mainly a queue for swapping between async operations, and a collection of abstracted services that the async operations can request, like network or disk i/o.

[deleted]
Post reply on HN