Earlier quoted context omitted.
Isolating a core and then pinning a single thread is the way to go to get both low latency and high throughput, sacrificing efficiency. This works fine on Linux, and common approach for trading systems where it’s fine to oversubscribe a bunch of cores for this type of stuff. The cores are mostly busy spinning and doing nothing, so it’s very inefficient in terms of actual work, but great for latency and throughput whe…
I just wish people who give this advice for 1 thread per core would "expand their reasoning" or "show the work". It's not blanket good advice for all things.
Io_uring, kTLS and Rust for zero syscall HTTPS server
111–120 of 173 posts
Re: Io_uring, kTLS and Rust for zero syscall HTTPS server
#112So, current status on async Rust - you need to understand: Futures, Pin, Waker, async runtimes, Send/Sync bounds, async trait objects, etc. C++20, coroutines. Go, goroutines. Java21+, virtual threads
The facts that Send/Sync bounds model are still relevant in all the other languages, the absence of Send/Sync just means it's easier to write subtly incorrect code.
Re: Io_uring, kTLS and Rust for zero syscall HTTPS server
#113Earlier quoted context omitted.
You have an inevitable overhead of managing the owned buffer when compared against simply passing mutable borrow to an already existing buffer. Imagine if `io::Read` APIs were constructed as `fn read(&mut self, buf: Vec ) -> io::Resul >`. Parity with synchronous programming is an explicit goal of Rust async declared many times (e.g. see here https://github.com/rust-lang/rust-project-goals/issues/105 ). I agree with y…
The problem is that the ring requires suitably locked memory which won't be subject to swapping, thus inherently forcing a different memory type if you want the extra-low-overhead/extra-scalable operation. It makes sense to ask the ring wrapper for memory that you can emplace your payload into before submitting the IO if you want to use zero-copy.
Re: Io_uring, kTLS and Rust for zero syscall HTTPS server
#114Earlier quoted context omitted.
No, the fundamental problem (in the context of io-uring) is that futures are managed by user code and can be dropped at any time. This often referred as "cancellation safety". Imagine a future has initialized completion-based IO with buffer which is part of the future state. User code can simply drop the future (e.g. if it was part of `select!`) and now we have a huge problem on our hands: the kernel will write into…
I really don’t understand this argument. If you force the user to transfer ownership of the buffer into the I/O subsystem, the system can make sure to transfer ownership of the buffer into the async runtime, not leaving it held within the cancellable future and the future returns that buffer which is given back when the completion is received from the kernel. What am I missing?
Re: Io_uring, kTLS and Rust for zero syscall HTTPS server
#115Earlier quoted context omitted.
Rust: Well yes. Rust does force you to understand the things, or it won't compile. It does have drawbacks. Go: goroutines are not async. And you can't understand goroutines without understanding channels. And channels are weirdly implemented in Go, where the semantics of edge cases, while well defined, are like rolling a D20 die if you try to reason from first principles. Go doesn't force you to understand things. I…
> Go: goroutines are not async Sure they are. The abstraction they provide is a synchronous API, but it's accomplished using an async runtime.
https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...
Re: Io_uring, kTLS and Rust for zero syscall HTTPS server
#116Earlier quoted context omitted.
There is, I think, an ownership model that Rust's borrow checker very poorly supports, and for lack of a better name, I've called it hot potato ownership. The basic idea is that you have a buffer which you can give out as ownership in the expectation that the person you gave it to will (eventually) give it back to you. It's a sort of non-lexical borrowing problem, and I very quickly discovered when trying to implemen…
This can be done with exclusively owned objects. That's how io_uring abstractions work in Rust – you give your (heap allocated) buffer to a buffer pool, and get it back when the operation is done. &mut references are exclusive and non-copyable, so the hot potato approach can even be used within their scope. But the problem in Rust is that threads can unwind/exit at any time, invalidating buffers living on the stack,…
In my case, I have code that essentially looks like this:
struct Parser {
state: ParserState
}
struct Subparser {
state: ParserState
}
impl Parser {
pub fn parse_something(&mut self) -> Subparser {
Subparse { state: self.state } // NOTE: doesn't work
}
}
impl Drop for Subparser {
fn drop(&mut self) {
parser.state = self.state; // NOTE: really doesn't work
}
}
Okay, I can make the first line work by changing Parser.state to be an Option instead and using Option::take (or std::mem::replace on a custom enum; going from an &mut T to a T is possible in a number of ways). But how do I give Subparser the ability to give its ParserState back to the original parser? If I could make Subparser take a lifetime and just have a pointer to Parser.state, I wouldn't even bother with half of this setup because I would just reach into the Parser directly, but that's not an option in this case. (The safe Rust option I eventually reached for is a oneshot channel, which is actually a lot of overhead for this case).It's the give-back portion of the borrow-to-give-back pattern that ends up being gnarly. I'm actually somewhat disappointed that the Rust ecosystem has in general given up on trying to build up safe pointer abstractions in the ecosystem, like doing use tracking for a pointed-to object. FWIW, a rough C++ implementation of what I would like to do is this:
template class HotPotato {
T *data;
HotPotato *borrowed_from = nullptr, *given_to = nullptr;
public:
T *get_data() {
// If we've given the data out, we can't use it at the moment.
return given_to ? nullptr : data;
}
std::unique_ptr> borrow() {
assert(given_to == nullptr);
auto *new_holder = new HotPotato();
new_holder->data = data;
new_holder->borrowed_from = this;
given_to = new_holder;
}
~HotPotato() {
if (given_to) {
given_to->borrowed_from = borrowed_from;
}
if (borrowed_from) {
borrowed_from->given_to = given_to;
} else {
delete data;
}
}
};Re: Io_uring, kTLS and Rust for zero syscall HTTPS server
#117So, current status on async Rust - you need to understand: Futures, Pin, Waker, async runtimes, Send/Sync bounds, async trait objects, etc. C++20, coroutines. Go, goroutines. Java21+, virtual threads
Note that C++ coroutines use heap allocation to avoid the problems that Pin is solving, which is a pretty big carve-out from the "zero overhead principle" that C++ usually aims for. The long development time of async traits has also been related to Rust not heap allocating futures. Whether that performance+portability-vs-complexity tradeoff is worth it for any given project is, of course, a different question.
Re: Io_uring, kTLS and Rust for zero syscall HTTPS server
#118Earlier quoted context omitted.
I really don’t understand this argument. If you force the user to transfer ownership of the buffer into the I/O subsystem, the system can make sure to transfer ownership of the buffer into the async runtime, not leaving it held within the cancellable future and the future returns that buffer which is given back when the completion is received from the kernel. What am I missing?
Requiring ownership transfer gives up on one of the main selling points of Rust, being able to verify reference lifetime and safety at compile time. If we have to give up on references then a lot of Rusts complexity no longer buys us anything.
self.buffer = io_read(self.buffer)?
This isn't much different than io_read(&mut self.buffer)?
since rust doesn't permit simultaneous access when a mutable reference is taken.Re: Io_uring, kTLS and Rust for zero syscall HTTPS server
#119Earlier quoted context omitted.
I don’t follow the zero copy argument. You pass in an owned buffer and get an owned buffer back out. There’s no copying happening here. It’s your claim that async is supposed to look like synchronous code but I don’t buy it. I don’t see why that’s a goal. Synchronous is an anachronistic software paradigm for a computer hardware architecture that never really existed (electronics are concurrent and asynchronous by nat…
You have an inevitable overhead of managing the owned buffer when compared against simply passing mutable borrow to an already existing buffer. Imagine if `io::Read` APIs were constructed as `fn read(&mut self, buf: Vec ) -> io::Resul >`. Parity with synchronous programming is an explicit goal of Rust async declared many times (e.g. see here https://github.com/rust-lang/rust-project-goals/issues/105 ). I agree with y…
To me it's pretty clear that parity in the issue referenced refers to equivalence parity - that is you can accomplish the tasks in some way, not that it's a drop-in replacement. I haven't seen anywhere suggested that async lets you write synchronous code without any changes, nor that integrating completion-style APIs with asynchronous will yield code that looks like synchronous. For one, completion-style APIs are for performance and performance APIs are rarely structured for simplicity but to avoid implicit costs hidden in common leaky (but simpler) abstractions. For another, completion-style APIs in synchronous programming ALSO looks different from epoll/select-like APIs, so I really don't understand the argument you're trying to make.
EDIT:
> You have an inevitable overhead of managing the owned buffer when compared against simply passing mutable borrow to an already existing buffer. Imagine if `io::Read` APIs were constructed as `fn read(&mut self, buf: Vec) -> io::Resul>`.
I'm imaging and I don't see a huge problem in terms of the overhead this implies. And you'd probably not necessarily take in a Vec directly but some I/O-specific type since such an API would be for performance.
Re: Io_uring, kTLS and Rust for zero syscall HTTPS server
#120Earlier quoted context omitted.
> don't use `let mut buf = [0u8; 16]; socket.read_all(&mut buf).await?;`. If you can't see why such arguments are bonkers, we don't have anything left to talk about. It doesn't seem bonkers to me. I know you already know these details, but spelling it out: If I'm using select/poll/epoll in C to do non-blocking reads of a socket, then yes I can use any old stack buffer to receive the bytes, because those are readiness…
>is that a failure of the async model This, 100%. Being really generous, it can be called a leaky model which is poorly compatible with completion-based APIs.
As someone else mentioned, what you really want is to ask io_uring to allocate the pages itself so that for reads it gives you pages that were allocated by the kernel to be filled directly by HW and then mapped into your userspace process without any copying by the kernel or any other SW layer involved.