31–40 of 53 posts
I worked with a greybeard that instilled in me that when we were about to do some RAID maintenance that we would always run sync twice. The second to make sure it immediately returns. And I added a third for my own anxiety.
You need to sync twice because Unix is dumb: "According to the standard specification (e.g., POSIX.1-2001), sync() schedules the writes, but may return before the actual writing is done." https://man7.org/linux/man-pages/man2/sync.2.html
> Submit the write to the primary file > Link fsync to that write (IOSQE_IO_LINK) > The fsync's completion queue entry only arrives after the write completes > Repeat for secondary file Wait, so the OS can re-order the fsync() to happen before the write request it is supposed to be syncing? Is there a citation or link to some code for that? It seems too ridiculous to be real. > O_DSYNC: Synchronous writes. Don't retu…
This is an io_uring-specific thing. It doesn't guarantee any ordering between operations submitted at the same time, unless you explicitly ask it to with the `IOSQE_IO_LINK` they mentioned.
Otherwise it's as if you called write() from one thread and fsync() from another, before waiting for the write() call to return. That obviously defeats the point of using fsync() so you wouldn't do that.
> If you call fsync(), [O_DSYNC] isn't needed correct? And if you use [O_DSYNC], then fsync() isn't needed right?
I believe you're right.
> Conclusion > A production-grade WAL isn't just code, it's a contract. I hate that I'm now suspicious of this formulation.
In what sense? The phrasing is just a generalization, production-grade anything needs consideration of the needs and goals of the project.
the language technique of negative parallel construction is a classic signal for AI writing
Earlier quoted context omitted.
You need to sync twice because Unix is dumb: "According to the standard specification (e.g., POSIX.1-2001), sync() schedules the writes, but may return before the actual writing is done." https://man7.org/linux/man-pages/man2/sync.2.html
Then how do you know the writes are done after the second sync?
Earlier quoted context omitted.
In what sense? The phrasing is just a generalization, production-grade anything needs consideration of the needs and goals of the project.
“ isn’t just , it’s ” is an AI smell.
Earlier quoted context omitted.
> Unix is dumb I don't know. Now async I/O is all the rage and that is the same idea.
If they had a sync() system call and a wait_for_sync_to_finish() system call then you'd be right. But they didn't have those.
Earlier quoted context omitted.
“ isn’t just , it’s ” is an AI smell.
Wouldn't that just be because the construction is common in the training materials, which means it's a common construction in human writing?
> Submit the write to the primary file > Link fsync to that write (IOSQE_IO_LINK) > The fsync's completion queue entry only arrives after the write completes > Repeat for secondary file Wait, so the OS can re-order the fsync() to happen before the write request it is supposed to be syncing? Is there a citation or link to some code for that? It seems too ridiculous to be real. > O_DSYNC: Synchronous writes. Don't retu…
> Wait, so the OS can re-order the fsync() to happen before the write request it is supposed to be syncing? Is there a citation or link to some code for that? It seems too ridiculous to be real. This is an io_uring-specific thing. It doesn't guarantee any ordering between operations submitted at the same time, unless you explicitly ask it to with the `IOSQE_IO_LINK` they mentioned. Otherwise it's as if you called wri…
Related: I would think that grouping your writes and then fsyncing rather than fsyncing every time would be more efficient but it looks like a previous commenter did some testing and that isn't always the case https://news.ycombinator.com/item?id=15535814