Live data from Hacker News

Io_uring, kTLS and Rust for zero syscall HTTPS server

blog.habets.se

111–120 of 173 posts

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#111
post #80

Earlier quoted context omitted.

Isolating a core and then pinning a single thread is the way to go to get both low latency and high throughput, sacrificing efficiency. This works fine on Linux, and common approach for trading systems where it’s fine to oversubscribe a bunch of cores for this type of stuff. The cores are mostly busy spinning and doing nothing, so it’s very inefficient in terms of actual work, but great for latency and throughput whe…

I just wish people who give this advice for 1 thread per core would "expand their reasoning" or "show the work". It's not blanket good advice for all things.

Check out Scylla and its underlying framework Seastar. They expand their reasoning and show the work.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#112
post #94
post #63

So, current status on async Rust - you need to understand: Futures, Pin, Waker, async runtimes, Send/Sync bounds, async trait objects, etc. C++20, coroutines. Go, goroutines. Java21+, virtual threads

The facts that Send/Sync bounds model are still relevant in all the other languages, the absence of Send/Sync just means it's easier to write subtly incorrect code.

Yeah the new typescript compiler that's written in Go crashed for me the other day because of some kind of concurrent modification. Java also has runtime checks for concurrent modification in its collections.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#113
post #99

Earlier quoted context omitted.

You have an inevitable overhead of managing the owned buffer when compared against simply passing mutable borrow to an already existing buffer. Imagine if `io::Read` APIs were constructed as `fn read(&mut self, buf: Vec ) -> io::Resul >`. Parity with synchronous programming is an explicit goal of Rust async declared many times (e.g. see here https://github.com/rust-lang/rust-project-goals/issues/105 ). I agree with y…

The problem is that the ring requires suitably locked memory which won't be subject to swapping, thus inherently forcing a different memory type if you want the extra-low-overhead/extra-scalable operation. It makes sense to ask the ring wrapper for memory that you can emplace your payload into before submitting the IO if you want to use zero-copy.

newpavlov seems to operate in a theoretical bubble. Until something like he describes is published publicly where we talk more concretely, it's not worth engaging. Requiring special buffers is very typical for any hardware offload with zero copy. It isn't a leaky abstraction. Async rust is well designed and I've not seen anything rival it. If there are problems it's in the libraries built on top like tokio.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#114

Earlier quoted context omitted.

No, the fundamental problem (in the context of io-uring) is that futures are managed by user code and can be dropped at any time. This often referred as "cancellation safety". Imagine a future has initialized completion-based IO with buffer which is part of the future state. User code can simply drop the future (e.g. if it was part of `select!`) and now we have a huge problem on our hands: the kernel will write into…

I really don’t understand this argument. If you force the user to transfer ownership of the buffer into the I/O subsystem, the system can make sure to transfer ownership of the buffer into the async runtime, not leaving it held within the cancellable future and the future returns that buffer which is given back when the completion is received from the kernel. What am I missing?

Requiring ownership transfer gives up on one of the main selling points of Rust, being able to verify reference lifetime and safety at compile time. If we have to give up on references then a lot of Rusts complexity no longer buys us anything.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#115
post #77

Earlier quoted context omitted.

Rust: Well yes. Rust does force you to understand the things, or it won't compile. It does have drawbacks. Go: goroutines are not async. And you can't understand goroutines without understanding channels. And channels are weirdly implemented in Go, where the semantics of edge cases, while well defined, are like rolling a D20 die if you try to reason from first principles. Go doesn't force you to understand things. I…

> Go: goroutines are not async Sure they are. The abstraction they provide is a synchronous API, but it's accomplished using an async runtime.

Yeah, in fact I'd argue that any abstraction that doesn't let you treat the work as sync is fundamentally broken.

https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#116
post #32

Earlier quoted context omitted.

There is, I think, an ownership model that Rust's borrow checker very poorly supports, and for lack of a better name, I've called it hot potato ownership. The basic idea is that you have a buffer which you can give out as ownership in the expectation that the person you gave it to will (eventually) give it back to you. It's a sort of non-lexical borrowing problem, and I very quickly discovered when trying to implemen…

This can be done with exclusively owned objects. That's how io_uring abstractions work in Rust – you give your (heap allocated) buffer to a buffer pool, and get it back when the operation is done. &mut references are exclusive and non-copyable, so the hot potato approach can even be used within their scope. But the problem in Rust is that threads can unwind/exit at any time, invalidating buffers living on the stack,…

Yes and no.

In my case, I have code that essentially looks like this:

   struct Parser {
     state: ParserState
   }
   struct Subparser {
     state: ParserState
   }
   impl Parser {
     pub fn parse_something(&mut self) -> Subparser {
       Subparse { state: self.state } // NOTE: doesn't work
     }
   }
   impl Drop for Subparser {
     fn drop(&mut self) {
       parser.state = self.state; // NOTE: really doesn't work
     }
   }
Okay, I can make the first line work by changing Parser.state to be an Option instead and using Option::take (or std::mem::replace on a custom enum; going from an &mut T to a T is possible in a number of ways). But how do I give Subparser the ability to give its ParserState back to the original parser? If I could make Subparser take a lifetime and just have a pointer to Parser.state, I wouldn't even bother with half of this setup because I would just reach into the Parser directly, but that's not an option in this case. (The safe Rust option I eventually reached for is a oneshot channel, which is actually a lot of overhead for this case).

It's the give-back portion of the borrow-to-give-back pattern that ends up being gnarly. I'm actually somewhat disappointed that the Rust ecosystem has in general given up on trying to build up safe pointer abstractions in the ecosystem, like doing use tracking for a pointed-to object. FWIW, a rough C++ implementation of what I would like to do is this:

  template  class HotPotato {
    T *data;
    HotPotato *borrowed_from = nullptr, *given_to = nullptr;

    public:
    T *get_data() {
      // If we've given the data out, we can't use it at the moment.
      return given_to ? nullptr : data;
    }
    std::unique_ptr> borrow() {
      assert(given_to == nullptr);
      auto *new_holder = new HotPotato();
      new_holder->data = data;
      new_holder->borrowed_from = this;
      given_to = new_holder;
    }

    ~HotPotato() {
      if (given_to) {
        given_to->borrowed_from = borrowed_from;
      }
      if (borrowed_from) {
        borrowed_from->given_to = given_to;
      } else {
        delete data;
      }
    }
  };

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#117
post #63

So, current status on async Rust - you need to understand: Futures, Pin, Waker, async runtimes, Send/Sync bounds, async trait objects, etc. C++20, coroutines. Go, goroutines. Java21+, virtual threads

Note that C++ coroutines use heap allocation to avoid the problems that Pin is solving, which is a pretty big carve-out from the "zero overhead principle" that C++ usually aims for. The long development time of async traits has also been related to Rust not heap allocating futures. Whether that performance+portability-vs-complexity tradeoff is worth it for any given project is, of course, a different question.

Quite a lot of work was done in Clang at least to elide allocations for coroutines where the compiler can see enough information.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#118
post #114

Earlier quoted context omitted.

I really don’t understand this argument. If you force the user to transfer ownership of the buffer into the I/O subsystem, the system can make sure to transfer ownership of the buffer into the async runtime, not leaving it held within the cancellable future and the future returns that buffer which is given back when the completion is received from the kernel. What am I missing?

Requiring ownership transfer gives up on one of the main selling points of Rust, being able to verify reference lifetime and safety at compile time. If we have to give up on references then a lot of Rusts complexity no longer buys us anything.

I'm not sure what you're trying to say, but the compile-time safety requirement isn't given up. It would look something like:

    self.buffer = io_read(self.buffer)?
This isn't much different than

    io_read(&mut self.buffer)?
since rust doesn't permit simultaneous access when a mutable reference is taken.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#119

Earlier quoted context omitted.

I don’t follow the zero copy argument. You pass in an owned buffer and get an owned buffer back out. There’s no copying happening here. It’s your claim that async is supposed to look like synchronous code but I don’t buy it. I don’t see why that’s a goal. Synchronous is an anachronistic software paradigm for a computer hardware architecture that never really existed (electronics are concurrent and asynchronous by nat…

You have an inevitable overhead of managing the owned buffer when compared against simply passing mutable borrow to an already existing buffer. Imagine if `io::Read` APIs were constructed as `fn read(&mut self, buf: Vec ) -> io::Resul >`. Parity with synchronous programming is an explicit goal of Rust async declared many times (e.g. see here https://github.com/rust-lang/rust-project-goals/issues/105 ). I agree with y…

The problem is that "parity" can be interpreted in different ways and you're choosing to interpret it in a way that doesn't seem to be communicated in the issue you referenced. In fact, the common definition of parity is something like "feature parity" meaning that you can do accomplish all the things you did before, even if it's not the same (e.g. MacOS has copy-paste feature parity even though it's a different shortcut or might work slightly differently from other operating systems). It rarely means "drop-in" parity where you don't have to do anything.

To me it's pretty clear that parity in the issue referenced refers to equivalence parity - that is you can accomplish the tasks in some way, not that it's a drop-in replacement. I haven't seen anywhere suggested that async lets you write synchronous code without any changes, nor that integrating completion-style APIs with asynchronous will yield code that looks like synchronous. For one, completion-style APIs are for performance and performance APIs are rarely structured for simplicity but to avoid implicit costs hidden in common leaky (but simpler) abstractions. For another, completion-style APIs in synchronous programming ALSO looks different from epoll/select-like APIs, so I really don't understand the argument you're trying to make.

EDIT:

> You have an inevitable overhead of managing the owned buffer when compared against simply passing mutable borrow to an already existing buffer. Imagine if `io::Read` APIs were constructed as `fn read(&mut self, buf: Vec) -> io::Resul>`.

I'm imaging and I don't see a huge problem in terms of the overhead this implies. And you'd probably not necessarily take in a Vec directly but some I/O-specific type since such an API would be for performance.

Re: Io_uring, kTLS and Rust for zero syscall HTTPS server

#120

Earlier quoted context omitted.

> don't use `let mut buf = [0u8; 16]; socket.read_all(&mut buf).await?;`. If you can't see why such arguments are bonkers, we don't have anything left to talk about. It doesn't seem bonkers to me. I know you already know these details, but spelling it out: If I'm using select/poll/epoll in C to do non-blocking reads of a socket, then yes I can use any old stack buffer to receive the bytes, because those are readiness…

>is that a failure of the async model This, 100%. Being really generous, it can be called a leaky model which is poorly compatible with completion-based APIs.

The leaky model is that you could ever receive into a stack buffer and you're arguing to persist this model. The reason it's leaky is that copying memory around is supremely expensive. But that's how the BSD socket API from the 90s works and btw something you can make work with async provided you're into memory copies. io_uring is a modern API that's for performance and that's why Rust libraries try to avoid memory copying within the internals. Supporting copying into the stack buffer with io_uring is very difficult to accomplish even in synchronous code. It's not a failure of async but a different programming paradigm altogether.

As someone else mentioned, what you really want is to ask io_uring to allocate the pages itself so that for reads it gives you pages that were allocated by the kernel to be filled directly by HW and then mapped into your userspace process without any copying by the kernel or any other SW layer involved.

Post reply on HN