Where can I get nanosecond latency streaming?
The screen in front of your face. That'll give you like 3ns latency.
Gazette: Cloud-native millisecond-latency streaming
11–20 of 53 posts
Re: Gazette: Cloud-native millisecond-latency streaming
#12Earlier quoted context omitted.
io_uring allows for async reads and writes to disk without forcing a thread pool or direct I/O. That alone makes it much more scalable for workloads that touch both the network and disk.
Doesn't the kernel use a thread pool to process the requests in the ring, because the kernel is still designed around blocking disk I/O?
No, most operations in the ring directly work asynchronously. The thread mechanism only exists as a fallback for combinations of operations and system configurations (e.g. filesystems) that don't support asynchronous operation.
Re: Gazette: Cloud-native millisecond-latency streaming
#13What's the use case for millisecond-latency streaming? HFT? Remotely driving heavy machinery? Anything else?
Re: Gazette: Cloud-native millisecond-latency streaming
#14What's the use case for millisecond-latency streaming? HFT? Remotely driving heavy machinery? Anything else?
Collaborative systems come to mind. If you edit a document and want to subscribe to changes from other nodes it is valuable to have very low latency.
Re: Gazette: Cloud-native millisecond-latency streaming
#15Earlier quoted context omitted.
Doesn't the kernel use a thread pool to process the requests in the ring, because the kernel is still designed around blocking disk I/O?
No, most operations in the ring directly work asynchronously. The thread mechanism only exists as a fallback for combinations of operations and system configurations (e.g. filesystems) that don't support asynchronous operation.
I don't know anything about the internals of io_uring and am genuinely curious how it works. Saying it "directly works asynchronously" doesn't mean anything though. When circular buffer requests are processed what thread is processing the request, how is that thread managed, and how does it manage blocking/unblocking when communicating with the storage device?
Re: Gazette: Cloud-native millisecond-latency streaming
#16What's the use case for millisecond-latency streaming? HFT? Remotely driving heavy machinery? Anything else?
I think it's less about guaranteed 1ms real time transactions and more about, like, it's just fast enough that you most likely don't have to worry about it introducing perceptible lag?
I'm working on a streaming audio thing and keeping latency low is a priority. I actually think I'll try Gazette, I just saw it now and it was one of those moments where it's like wait I go to Hacker News to waste time but this is quite exactly what I've been wanting in so many ways.
I'll use it for Ogg/Opus media streams, transcription results, chat events, LLM inferences...
I really like the byte-indexed append-only blob paradigm backed by object storage. It feels kind of like Unix as a distributed streaming system.
Other streaming data gadgets like Kafka always feel a bit uncomfortable and annoying to me with their idiosyncratic record formats and topic hierarchies and whatnot... I always wanted something more low level and obvious...
Re: Gazette: Cloud-native millisecond-latency streaming
#17Earlier quoted context omitted.
No, most operations in the ring directly work asynchronously. The thread mechanism only exists as a fallback for combinations of operations and system configurations (e.g. filesystems) that don't support asynchronous operation.
I don't know anything about the internals of io_uring and am genuinely curious how it works. Saying it "directly works asynchronously" doesn't mean anything though. When circular buffer requests are processed what thread is processing the request, how is that thread managed, and how does it manage blocking/unblocking when communicating with the storage device?
Internally, many parts of the Linux kernel operate asynchronously: they queue up a request with some subsystem (e.g. a hardware device), and get an event delivered when the request is completed. In such cases, io_uring can enqueue such a request, and complete it when receiving the event, without needing to use a thread to block waiting for it.
See, for instance, https://lpc.events/event/11/contributions/901/attachments/78... slide 5 (though more has happened since then). io_uring will first see if it has everything needed to do the operation immediately, if not it'll queue a request in some cases (e.g. direct I/O, or buffered I/O in some cases). The thread pool is the last fallback, which always works if nothing else does.
https://lwn.net/Articles/821274/ talks about making async buffered reads work, for instance.
Re: Gazette: Cloud-native millisecond-latency streaming
#18I feel a bit paralyzed by Fear Of Missing Io_Uring. There's so much awesome streaming stuff about (RisingWave, Materialize, NATS, DataFusion, Velox, neat upstarts like Iggy, many more), but it all feels built on slower legacy system libraries. It's not heavily used yet, but Rust has a bunch of fairly high visibility efforts. Situation sort of feels similar with http3, where the problem is figuring out what to pick. h…
I seem to be missing context for this reply. Why do you need io_uring?
You don't need io_uring. For many workloads being slow & inefficient is acceptable, isn't awful. But gee I'd rather start from a modern baseline that has high levels of mechanistic sympathy with the hardware, where things like network & io work can be done in an efficient async manner.
Why do I need io_uring? Because it sounds awful and unhackerly to suffer living in a much lesser worse world.
Re: Gazette: Cloud-native millisecond-latency streaming
#19Re: Gazette: Cloud-native millisecond-latency streaming
#20Earlier quoted context omitted.
I seem to be missing context for this reply. Why do you need io_uring?
You don't need io_uring. For many workloads being slow & inefficient is acceptable, isn't awful. But gee I'd rather start from a modern baseline that has high levels of mechanistic sympathy with the hardware, where things like network & io work can be done in an efficient async manner. Why do I need io_uring? Because it sounds awful and unhackerly to suffer living in a much lesser worse world.
Mechanical sympathy is understanding the system, not using the shiniest thing. If you want low latency processing of one event at a time, you are either going to burn an entire core spinning or you are going to do a syscall for each operation. The io_uring syscalls are not especially fast — they get their awesomeness by doing, potentially, a whole lot of work per operation. And, for some use cases, by having a superior async IO model.
But if you actually just want read(), then call read().