31–40 of 123 posts
ZFS fsync will not fail, although it could end up waiting forever when a pool faults due to hardware failures:
https://papers.freebsd.org/2024/asiabsdcon/norris_openzfs-fs...
Earlier quoted context omitted.
Not really, there's been lots of APIs that have improved on the POSIX model. The kind of model I prefer is something based on atomicity. Most applications can get by with file-level atomicity--make whole file read/writes atomic with a copy-on-write model, and you can eliminate whole classes of filesystem bugs pretty quickly. (Note that something like writeFileAtomic is already a common primitive in many high-level fi…
> make whole file read/writes atomic with a copy-on-write model, I have many files that are several GB. Are you sure this is a good idea? What if my application only requires best effort? > eliminate whole classes of filesystem bugs pretty quickly. Block level deduplication is notoriously difficult. > where the only kind of write allowed is to atomically append a chunk of data to the file Which sounds good until you…
But even then, doing atomic writes of multi gigabyte files doesn’t sound that hard to implement efficiently. Just write to disk first and update the metadata atomically at the end. Or whenever you choose to as a programmer.
The downside is that, when overwriting, you’ll need enough free space to store both the old and new versions of your data. But I think that’s usually a good trade off.
It would allow all sorts of useful programs to be written easily - like an atomic mode for apt, where packages either get installed or not installed. But they can’t be half installed.
No mention on ntfs and windows keywords in the article, for those interested.
Is that because the windows APIs are better? Or because businesses build their embedded systems/servers with Windows?
I can't say the Win32 File API is "pretty", but it's also an abstraction, like the .NET File Class is. And if you touch the NT API, you're naughty.
On Linux and macOS you use the same API, just the backends are different if you want async (epoll [blocking async] on Linux, kqueue on macOS).
Earlier quoted context omitted.
By the way, LMDB's main developer Howard Chu responded to the paper. He said, > They report on a single "vulnerability" in LMDB, in which LMDB depends on the atomicity of a single sector 106-byte write for its transaction commit semantics. Their claim is that not all storage devices may guarantee the atomicity of such a write. While I myself filed an ITS on this very topic a year ago, http://www.openldap.org/its/inde…
This assumption was wrong for Intel Optane memory. Power loss could cut the data stream anywhere in the middle. (Note: the DIMM nonvolatile memory version)
The crash-consistency problem is very different than the durability of real synchronous writes problem. There are some storage devices which will lie about synch writes, sometimes hoping that a backup battery will allow them to complete those write.
System crashes are inevitable, use things like write ahead logs depending on need etc... No storage API will get rid of all system crashes and yes even apple games the system by disabling real sync writes, so that will always be a battle.
I don't get it. The only times I've had problems with filesystem corruption in the past few decades was with a hardware problem, and said hardware was quickly replaced. FAT family has been perfectly fine while I've encountered corruption on every other FS including NTFS, exFAT, and the ext* family. Meanwhile you can read plenty of stories of others having the exact opposite experience. If you keep losing data to powe…
I keep telling my users to make sure to plug their phones in before the battery dies, but for some reason they keep forgetting...
Earlier quoted context omitted.
POSIX is also so old and essential that it's hard to imagine an alternative.
Not really, there's been lots of APIs that have improved on the POSIX model. The kind of model I prefer is something based on atomicity. Most applications can get by with file-level atomicity--make whole file read/writes atomic with a copy-on-write model, and you can eliminate whole classes of filesystem bugs pretty quickly. (Note that something like writeFileAtomic is already a common primitive in many high-level fi…
https://github.com/openzfs/zfs/blob/34205715e1544d343f9a6414...
Writes on ZFS cease to be atomic around approximately 32MB in size if I read the code correctly.
> they found that every single piece of software they tested except for SQLite in one particular mode had at least one bug. This is why whenever I need to persist any kind of state to disk, SQLite is the first tool I reach for. Filesystem APIs are scary, but SQLite is well-behaved. Of course, it doesn't always make sense to do that, like the dropbox use case.
Before becoming too overconfident in SQLite note that Rebello et al. ( https://ramalagappan.github.io/pdfs/papers/cuttlefs.pdf ) tested SQLite (along with Redis, LMDB, LevelDB, and PostgreSQL) using a proxy file system to simulate fsync errors and found that none of them handled all failure conditions safely. In practice I believe I've seen SQLite databases corrupted due to what I suspect are two main causes: 1. The…
https://lists.openldap.org/hyperkitty/list/openldap-devel@op...
I'm pretty sure that's not where I originally saw his comments. I remember his criticisms being a little more pointed. Although I guess "This is a bunch of academic speculation, with a total absence of real world modeling to validate the failure scenarios they presented" is pretty pointed.
Earlier quoted context omitted.
This assumption was wrong for Intel Optane memory. Power loss could cut the data stream anywhere in the middle. (Note: the DIMM nonvolatile memory version)
consumer Optane were not "power loss protected", that is every different than not honoring a requested a synchronous write. The crash-consistency problem is very different than the durability of real synchronous writes problem. There are some storage devices which will lie about synch writes, sometimes hoping that a backup battery will allow them to complete those write. System crashes are inevitable, use things like…
Earlier quoted context omitted.
> make whole file read/writes atomic with a copy-on-write model, I have many files that are several GB. Are you sure this is a good idea? What if my application only requires best effort? > eliminate whole classes of filesystem bugs pretty quickly. Block level deduplication is notoriously difficult. > where the only kind of write allowed is to atomically append a chunk of data to the file Which sounds good until you…
It doesn’t have to be one or the other. Developers could decide by passing flags to open. But even then, doing atomic writes of multi gigabyte files doesn’t sound that hard to implement efficiently. Just write to disk first and update the metadata atomically at the end. Or whenever you choose to as a programmer. The downside is that, when overwriting, you’ll need enough free space to store both the old and new versio…
Earlier quoted context omitted.
> why the file API so hard to use that even experts make mistakes? I think the short answer is that the APIs are bad. The POSIX fs APIs and associated semantics are so deeply entrenched in the software ecosystem (both at the OS level, and at the application level) that it's hard to move away from them.
POSIX is also so old and essential that it's hard to imagine an alternative.