Async I/O on Linux in databases • Hacker News Board

Re: Async I/O on Linux in databases

#11

Post by Manuel_D » Sun, Jul 20, 2025, 8:47 AM UTC

gsliepen wrote: post #7

ta8645 wrote: post #5

Earlier quoted context omitted.

It's really not clear in the article. But I _think_ the gains are to be had because you can do the in-memory updating during the time that the WAL is being written to disk (rather than waiting for it to flush before proceeding). So I'm guessing the protocol as presented, is actually missing a key step: Write intent record (async) Perform operation in memory Write completion record (async) * * Wait for intent and comp…

But this makes me wonder how it works when there are concurrent requests. What if a second thread requests data that is being written to memory by the first thread? Shouldn't it also wait for both the write intent record and completion record having been flushed to disk? Otherwise you could end up with a query that returns data that after a crash won't exist anymore.

It's not the write ahead log that prevents that scenario, it's transaction isolation. And note that the more permissive isolation levels offered by Postgres, for example, do allow that failure mode to occur.

Re: Async I/O on Linux in databases

#12

Post by [deleted] » Sun, Jul 20, 2025, 8:59 AM UTC

JasonSage wrote: post #8

jmpman wrote: post #6

“Write intent record (async) Perform operation in memory Write completion record (async) Return success to client During recovery, I only apply operations that have both intent and completion records. This ensures consistency while allowing much higher throughput. “ Does this mean that a client could receive a success for a request, which if the system crashed immediately afterwards, when replayed, wouldn’t necessari…

As best I can tell, the author understands that the async write-ahead fails to be a guarantee where the sync one does… then turns their async write into two async writes… but there’s still no guarantee comparable to the synchronous version. So I fail to see how the two async writes are any guarantee at all. It sounds like they just happen to provide better consistency than the one async write because it forces an arb…

[deleted]

Re: Async I/O on Linux in databases

#13

Post by ozgrakkurt » Sun, Jul 20, 2025, 9:04 AM UTC

Great to see someone going into this. I wanted to do a simple LSM tree using io_uring in Zig for some time but couldn't get into it yet.

I always use this approach for crash-resistance:

- Append to the data (WAL) file normally.

- Have a seperate small file that is like a hash + length for WAL state.

- First append to WAL file.

- Start fsync call on the WAL file, create a new hash/length file with different name and fsync it in parallel.

- Rename the length file onto the real one for making sure it is fully atomic.

- Update in-memory state to reflect the files and return from the write function call.

Curious if anyone knows tradeoffs between this and doing double WAL. Maybe doing fsync on everything is too slow to maintain fast writes?

I learned about append/rename approach from this article in case anyone is interested:

- https://discuss.hypermode.com/t/making-badger-crash-resilien...

- https://research.cs.wisc.edu/adsl/Publications/alice-osdi14....

Re: Async I/O on Linux in databases

#14

Post by tobias3 » Sun, Jul 20, 2025, 9:26 AM UTC

I don't get this. How can two(+) WAL operations be faster than one (double the sync IOPS)?

I think this database doesn't have durability at all.

Re: Async I/O on Linux in databases

#15

Post by toolslive » Sun, Jul 20, 2025, 9:29 AM UTC

ozgrakkurt wrote: post #13

Great to see someone going into this. I wanted to do a simple LSM tree using io_uring in Zig for some time but couldn't get into it yet. I always use this approach for crash-resistance: - Append to the data (WAL) file normally. - Have a seperate small file that is like a hash + length for WAL state. - First append to WAL file. - Start fsync call on the WAL file, create a new hash/length file with different name and f…

it's possible to unify the WAL and the tree. There are some append only B-tree implementations. https://github.com/Incubaid/baardskeerder fe.

Re: Async I/O on Linux in databases

#16

Post by m11a » Sun, Jul 20, 2025, 9:39 AM UTC

JasonSage wrote: post #8

jmpman wrote: post #6

“Write intent record (async) Perform operation in memory Write completion record (async) Return success to client During recovery, I only apply operations that have both intent and completion records. This ensures consistency while allowing much higher throughput. “ Does this mean that a client could receive a success for a request, which if the system crashed immediately afterwards, when replayed, wouldn’t necessari…

As best I can tell, the author understands that the async write-ahead fails to be a guarantee where the sync one does… then turns their async write into two async writes… but there’s still no guarantee comparable to the synchronous version. So I fail to see how the two async writes are any guarantee at all. It sounds like they just happen to provide better consistency than the one async write because it forces an arb…

Yeah, I feel like I’m missing the point of this. The original purpose of the WAL was for recovery, so WAL entries are supposed to be flushed to disk.

Seems like OP’s async approach removes that, so there’s no durability guarantee, so why even maintain a WAL to begin with?

Re: Async I/O on Linux in databases

#17

Post by avinassh » Sun, Jul 20, 2025, 9:48 AM UTC

ta8645 wrote: post #5

tlb wrote: post #3

The recovery process is to "only apply operations that have both intent and completion records." But then I don't see the point of logging the intent record separately. If no completion is logged, the intent is ignored. So you could log the two together. Presumably the intent record is large (containing the key-value data) while the completion record is tiny (containing just the index of the intent record). Is the po…

It's really not clear in the article. But I _think_ the gains are to be had because you can do the in-memory updating during the time that the WAL is being written to disk (rather than waiting for it to flush before proceeding). So I'm guessing the protocol as presented, is actually missing a key step: Write intent record (async) Perform operation in memory Write completion record (async) * * Wait for intent and comp…

    * * Wait for intent and completion to be flushed to disk * *

if you wait for both to complete, then how it can be faster than doing a single IO?

Re: Async I/O on Linux in databases

#18

Post by avinassh » Sun, Jul 20, 2025, 9:53 AM UTC

I don't get this scheme at all. The protocol violates durability, because once the client receives success from server, it should be durable. However, completion record is async, it is possible that it never completes and server crashes.

During recovery, since the server applies only the operations which have both records, you will not recover a record which was successful to the client.

Re: Async I/O on Linux in databases

#19

Post by avinassh » Sun, Jul 20, 2025, 9:54 AM UTC

toolslive wrote: post #15

ozgrakkurt wrote: post #13

Great to see someone going into this. I wanted to do a simple LSM tree using io_uring in Zig for some time but couldn't get into it yet. I always use this approach for crash-resistance: - Append to the data (WAL) file normally. - Have a seperate small file that is like a hash + length for WAL state. - First append to WAL file. - Start fsync call on the WAL file, create a new hash/length file with different name and f…

it's possible to unify the WAL and the tree. There are some append only B-tree implementations. https://github.com/Incubaid/baardskeerder fe.

There are also CoW B Trees not entirely similar, but kinda same.

Re: Async I/O on Linux in databases

#20

Post by nephalegm » Sun, Jul 20, 2025, 10:27 AM UTC

m11a wrote: post #16

JasonSage wrote: post #8

Earlier quoted context omitted.

As best I can tell, the author understands that the async write-ahead fails to be a guarantee where the sync one does… then turns their async write into two async writes… but there’s still no guarantee comparable to the synchronous version. So I fail to see how the two async writes are any guarantee at all. It sounds like they just happen to provide better consistency than the one async write because it forces an arb…

Yeah, I feel like I’m missing the point of this. The original purpose of the WAL was for recovery, so WAL entries are supposed to be flushed to disk. Seems like OP’s async approach removes that, so there’s no durability guarantee, so why even maintain a WAL to begin with?

Reading through the article it’s explained in the recovery process. He reads the intent log entries and the completion entries and only applies them if they both exist.

So there is no guarantee that operations are committed by virtue of not being acknowledged to the application (asynchronous) the recovery replay will be consistent.

I could see it would be problematic for any data where the order of operations is important, but that’s the trade off for performance. This does seem to be an improvement to ensure asynchronous IO will always result in a consistent recovery.