Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Io_uring, kTLS and Rust for zero syscall HTTPS server

blog.habets.se

Post reply on HN
Print view View original

71–80 of 173 posts

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#71

Post by tliltocatl »

kev009 wrote: post #12
bmcahren wrote: post #6

This was a good read and great work. Can't wait to see the performance tests. Your write up connected some early knowledge from when I was 11 where I was trying to set up a database/backend and was finding lots of cgi-bin online. I realize now those were spinning up new processes with each request https://en.wikipedia.org/wiki/Common_Gateway_Interface I remember when sendfile became available for my large gaming foru…

It wasn't just CGI, every HTTP session was commonly a forked copy of the entire server in the CERN and Apache lineage! Apache gradually had better answers, but their API with common addons made it a bit difficult to transition so webservers like nginx took off which are built closer to the architecture in the article with event driven I/O from the beginning.

That's because Unix API used to assume fork() is extremely cheap. Threads were ugly performance hack second-class citizens - still are in some ways. This was indeed true on PDP-11 (just copy a <64KB disk file!), but as address spaces grew, it became prohibitively expensive to copy page tables, so programmers turned to multithreading. At then multicore CPUs became the norm, and multithreading on multicore CPUs meant any kind of copy-on-write required TLB shootdown, making fork() even more expensive. VMS (and its clone known as Windows NT) did it right from the start - processes are just resource containers, units execution are threads and all IO is async. But being technically superior doesn't outweighs the disadvantage of being proprietary.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#72

Post by ozgrakkurt »

bullen wrote: post #40

So far everything after epoll that I have compared with falls short. So to reimplement my foundation (with all the bugs) will not be worth it. I will however compare Javas NIO (epoll) with the new Virtual Threads IO (without pinning). http://github.com/tinspin/rupy

This wiki page might be useful for anyone that is looking into this

https://github.com/axboe/liburing/wiki/io_uring-and-networki...

Also there is napi support in uring which uses polled io on sockets instead of interrupt based io from what I understand. You can see examples using it in liburing github

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#73

Post by [deleted] »

johncolanduoni wrote: post #52
aliceryhl wrote: post #50

Earlier quoted context omitted.

> IIRC Alice from the tokio team also suggested there hasn't been much interest in pushing through these difficulties more recently, as the current performance is "good enough". Well, I think there is interest, but mostly for file IO. For file IO, the situation is pretty simple. We already have to implement that using spawn_blocking, and spawn_blocking has the exact same buffer challenges as io_uring does, so transla…

This covers probably 90% of the usefulness of io_uring for non-niche applications. Its original purpose was doing buffered async file IO without a bunch of caveats that make it effectively useless. The biggest speed up I’ve found with it is ‘stat’ing large sets of files in the VFS cache. It can literally be 50x faster at that, since you can do 1000 files with a single systemcall and the data you need from the kernel…

[deleted]

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#74

Post by newpavlov »

withoutboats3 wrote: post #68
newpavlov wrote: post #42

Earlier quoted context omitted.

This actually one of my many gripes about Rust async and why I consider it a bad addition to the language in the long term. The fundamental problem is that rust async was developed when epoll was dominant (and almost no one in the Rust circles cared about IOCP) and it has heavily influenced the async design (sometimes indirectly through other languages). Think about it for a second. Why do we not have this problem wi…

genuinely so sad to me that you are still grinding this axe. if your fantasy design works so much better - go build it then!

Deal with it. Async is my greatest disappointment in the otherwise mostly stellar language. And I will continue to argue strongly against it.

After Rust has raised the level of quality and expectations to such great level, async feels like 3 steps back with all those arguments "you are holding it wrong", footguns, and piles of hacks. And this sentiment is shared by many others. It's really disappointing to see how many resources are getting sunk into the flawed async model by both the language and the ecosystem developers.

>go build it then

I did build it and it's in the process of being adopted into a proprietary database (theoretically a prime use-case for async Rust). Sadly, because I don't have ways to change the language and the compiler, it has obvious limitations (and generally it can be called unsound, especially around thread locals). It works for our project only because we have a tightly controlled code base. In future I plan to create a custom "green-thread" fork of `std` to ease limitations a bit. Because of the limitations (and the proprietary nature of the project) it is unlikely to be published as an open source project.

Amusingly, during online discussions I've seen other unrelated people who done similar stuff.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#76

Post by dpeckett »

johncolanduoni wrote: post #52
aliceryhl wrote: post #50

Earlier quoted context omitted.

> IIRC Alice from the tokio team also suggested there hasn't been much interest in pushing through these difficulties more recently, as the current performance is "good enough". Well, I think there is interest, but mostly for file IO. For file IO, the situation is pretty simple. We already have to implement that using spawn_blocking, and spawn_blocking has the exact same buffer challenges as io_uring does, so transla…

This covers probably 90% of the usefulness of io_uring for non-niche applications. Its original purpose was doing buffered async file IO without a bunch of caveats that make it effectively useless. The biggest speed up I’ve found with it is ‘stat’ing large sets of files in the VFS cache. It can literally be 50x faster at that, since you can do 1000 files with a single systemcall and the data you need from the kernel…

For TCP streams syscall overhead isn't a big issue really, you can easily transfer large chunks of data in each write(). If you have TCP segmentation offload available you'll have no serious issues pushing 100gbit/s. Also if you are sending static content don't forget sendfile().

UDP is a whole another kettle of fish, get's very complicated to go above 10gbit/s or so. This is a big part of why QUIC really struggles to scale well for fat pipes [1]. sendmmsg/recvmmsg + UDP GRO/GSO will probably get you to ~30gbit/s but beyond that is a real headache. The issue is that UDP is not stream focused so you're making a ton of little writes and the kernel networking stack as of today does a pretty bad job with these workloads.

FWIW even the fastest QUIC implementations cap out at Had a good fight writing a ~20gbit userspace UDP VPN recently. Ended up having to bypass the kernels networking stack using AF_XDP [3].

I'm available for hire btw, if you've got an interesting networking project feel free to reach out.

1. https://arxiv.org/abs/2310.09423

2. https://microsoft.github.io/msquic/

3. https://github.com/apoxy-dev/icx/blob/main/tunnel/tunnel.go

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#77

Post by ori_b »

thomashabets2 wrote: post #66
npalli wrote: post #63

So, current status on async Rust - you need to understand: Futures, Pin, Waker, async runtimes, Send/Sync bounds, async trait objects, etc. C++20, coroutines. Go, goroutines. Java21+, virtual threads

Rust: Well yes. Rust does force you to understand the things, or it won't compile. It does have drawbacks. Go: goroutines are not async. And you can't understand goroutines without understanding channels. And channels are weirdly implemented in Go, where the semantics of edge cases, while well defined, are like rolling a D20 die if you try to reason from first principles. Go doesn't force you to understand things. I…

> Go: goroutines are not async

Sure they are. The abstraction they provide is a synchronous API, but it's accomplished using an async runtime.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#78

Post by butterisgood »

Where do people get the idea that one thread per core is correct on a system that deals with time slices?

In my experience “oversubscribing” threads to cores (more threads than cores) provides a wall-clock time benefit.

I think one thread per core would work better without preemptive scheduling.

But then we aren’t talking about Unix.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#79

Post by tele_ski »

IX-103 wrote: post #64
alfiedotwtf wrote: post #48

Earlier quoted context omitted.

In my universe, `let` wouldn’t exist… instead there would only be 3 ways to declare variables: 1. global my_global_var: GlobalType = … 2. heap my_heap_var: HeapType = … 3. stack my_stack_var: StackType = … Global types would need to implement a global trait to ensure mutual exclusion (waves hands). So by having the location of allocation in the type itself, we no longer have to do boxing mental gymnastics

Doesn't Rust do this? `let` is always on the stack. If you want to allocate on the heap then you need a Box. So `let foo = Box::new(MyFoo::default ())` creates a Box on the stack that points to a MyFoo on the heap. So MyFoo is a stack type and Box is a heap type. Or do you think there is value in defining MyFooStack and MyFooHeap separately to support both use cases?

The suggestion is c# class vs struct basically, with explicit globals which are just class with synchronization

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#80

Post by gorset »

butterisgood wrote: post #78

Where do people get the idea that one thread per core is correct on a system that deals with time slices? In my experience “oversubscribing” threads to cores (more threads than cores) provides a wall-clock time benefit. I think one thread per core would work better without preemptive scheduling. But then we aren’t talking about Unix.

Isolating a core and then pinning a single thread is the way to go to get both low latency and high throughput, sacrificing efficiency.

This works fine on Linux, and common approach for trading systems where it’s fine to oversubscribe a bunch of cores for this type of stuff. The cores are mostly busy spinning and doing nothing, so it’s very inefficient in terms of actual work, but great for latency and throughput when you need it.

Post reply on HN