Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

High-Performance DBMSs with io_uring: When and How to use it

arxiv.org

Post reply on HN
Print view View original

31–40 of 50 posts

Re: High-Performance DBMSs with io_uring: When and How to use it

#31

Post by Asmod4n »

georgyo wrote: post #27
Asmod4n wrote: post #24

Earlier quoted context omitted.

It’s manageable with eBPF instead of seccomp so one has to adapt to that. Should be doable.

Maybe not so doable. The whole point of io_uring is to reduce syscalls. So you end up just three. io_uring_setup, io_uring_register, io_uring_enter There is now a memory buffer that the user space and the kernel is reading, and with that buffer you can _always_ do any syscall that io_uring supports. And things like strace, eBPF, and seccomp cannot see the actual syscalls that are being called in that memory buffer. A…

Ain’t eBPF hooks there so you can limit what a cgroup/process can do, not matter what API it’s calling. Like disallowing opening files or connecting sockets altogether.

Re: High-Performance DBMSs with io_uring: When and How to use it

#32

Post by melhindi »

bikelang wrote: post #19

Do any cloud hosting providers make io_uring available? Or are they all blocking it due to perceived security risks? I think I have a killer use case and would love to experiment - but it’s unclear who even makes this available.

Adding to the answer from cptnntsoobv: All major cloud providers (AWS, Azure, GCP) and also smaller providers (e.g., Hetzner) support io_uring on VMs. We ran some tests on AWS, for example. The experiments in the paper were done on our lab infrastructure to test io_uring on 400G NICs (which are not yet widely available in the cloud)

Re: High-Performance DBMSs with io_uring: When and How to use it

#33

Post by actionfromafar »

georgyo wrote: post #27
Asmod4n wrote: post #24

Earlier quoted context omitted.

It’s manageable with eBPF instead of seccomp so one has to adapt to that. Should be doable.

Maybe not so doable. The whole point of io_uring is to reduce syscalls. So you end up just three. io_uring_setup, io_uring_register, io_uring_enter There is now a memory buffer that the user space and the kernel is reading, and with that buffer you can _always_ do any syscall that io_uring supports. And things like strace, eBPF, and seccomp cannot see the actual syscalls that are being called in that memory buffer. A…

So io_uring is like transactions in sql but for syscalls?

Re: High-Performance DBMSs with io_uring: When and How to use it

#34

Post by LAC-Tech »

to_ziegler wrote: post #29
LAC-Tech wrote: post #22

NVMe passthrough skips abstractions. To access NVMe de- vices directly, io_uring provides the OP_URING_CMD opcode, which issues native NVMe commands via the kernel to device queues. By bypassing the generic storage stack, passthrough reduces software- layer overhead and per-I/O CPU cost. This yields an additional 20% gain, increasing throughput to 300 k tx/s (Figure 5, +Passthru). Which userspace libraries support th…

Great point. It is indeed the case that most io_uring libraries are lagging behind liburing. There are two main reasons for this: (1) io_uring development is moving very quickly (see the linked figure in another comment), and (2) liburing is maintained by Axboe himself and is therefore tightly coupled to io_uring’s ongoing development. One pragmatic "solution" is to wrap liburing with FFI bindings and maintain those…

That's what liburing_rs[1] is doing.

I am foolishly trying to make pure rust bindings myself [2].

[1] https://docs.rs/axboe-liburing/

[2] https://docs.rs/hringas

But yeah the liburing test suite is massive, way more than anyone else has.

Re: High-Performance DBMSs with io_uring: When and How to use it

#35

Post by mjasny »

jclulow wrote: post #23
CoolCold wrote: post #18

> Figure 9: Durable writes with io_uring. Left: Writes and fsync are issued via io_uring or manually linked in the application. Right: Enterprise SSDs do not require fsync after writes. This sounds strange to me, of not requiring fsync. I may be wrong, but if it was meant that Enterprise SSDs have buffers and power-failure safety modes which works fine without explicit fsync, I think it's too optimistic view here.

Yeah that's just flat out not correct. If you're writing through a file system or the buffer cache and you don't fsync, there is no guarantee your data will still be there after, say, a power loss or a system panic. There's no guarantee it's even been passed to the device at all when an asynchronous write returns.

Yes, for file systems these statements are true.

However, in our experiments (including Figure 9), we bypass the page cache and issue writes using O_DIRECT to the block device. In this configuration, write completion reflects device-level persistence. For consumer SSDs without PLP, completions do not imply durability and a flush is still required.

> "When an SSD has Power-Loss Protection (PLP) -- for example, a supercapacitor that backs the write-back cache -- then the device's internal write-back cache contents are guaranteed durable even if power is lost. Because of this guarantee, the storage controller does not need to flush the cache to media in a strict ordering or slow way just to make data persistent." (Won et al., FAST 2018) https://www.usenix.org/system/files/conference/fast18/fast18...

We will make this more explicit in the next revision. Thanks.

Re: High-Performance DBMSs with io_uring: When and How to use it

#36

Post by to_ziegler »

LAC-Tech wrote: post #34
to_ziegler wrote: post #29

Earlier quoted context omitted.

Great point. It is indeed the case that most io_uring libraries are lagging behind liburing. There are two main reasons for this: (1) io_uring development is moving very quickly (see the linked figure in another comment), and (2) liburing is maintained by Axboe himself and is therefore tightly coupled to io_uring’s ongoing development. One pragmatic "solution" is to wrap liburing with FFI bindings and maintain those…

That's what liburing_rs[1] is doing. I am foolishly trying to make pure rust bindings myself [2]. [1] https://docs.rs/axboe-liburing/ [2] https://docs.rs/hringas But yeah the liburing test suite is massive, way more than anyone else has.

Pretty nice library! Personally, I think having a language-native library is great. The only downside is that it becomes a long-term commitment if it needs to stay in sync with the development of io_uring. I wonder whether it would be a good idea to provide C bindings from the language-specific library, so it can be more easily integrated and tested into the liburing test suite with minimal effort.

Re: High-Performance DBMSs with io_uring: When and How to use it

#37

Post by charcircuit »

hayd wrote: post #15
topspin wrote: post #7

Earlier quoted context omitted.

In your high level "You might not want to use it if" points, you mention Docker but not why, and that's odd. I happen to know why: io_uring syscalls are blocked by default in Docker, because io_uring is a large surface area for attacks, and this has proven to be a real problem in practice. Others won't know this, however. They also won't know that io_uring is similarly blocked in widely used cloud sandboxes, Android,…

Is this something likely to ever change?

It already did with the io_uring worker rewrite in 5.12 (2021) which made it much safer.

https://github.com/axboe/liburing/discussions/1047

Re: High-Performance DBMSs with io_uring: When and How to use it

#38

Post by LAC-Tech »

to_ziegler wrote: post #36
LAC-Tech wrote: post #34

Earlier quoted context omitted.

That's what liburing_rs[1] is doing. I am foolishly trying to make pure rust bindings myself [2]. [1] https://docs.rs/axboe-liburing/ [2] https://docs.rs/hringas But yeah the liburing test suite is massive, way more than anyone else has.

Pretty nice library! Personally, I think having a language-native library is great. The only downside is that it becomes a long-term commitment if it needs to stay in sync with the development of io_uring. I wonder whether it would be a good idea to provide C bindings from the language-specific library, so it can be more easily integrated and tested into the liburing test suite with minimal effort.

woah that reverse uno of making C bindings from my rust bindings so I can piggy back on the awesome liburing tests is really clever :) thanks for that.
Post reply on HN