Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Working with Files Is Hard (2019)

danluu.com

Post reply on HN
Print view View original

71–80 of 123 posts

Re: Working with Files Is Hard (2019)

#71

Post by nickelpro »

continuational wrote: post #4

> Pillai et al., OSDI’14 looked at a bunch of software that writes to files, including things we'd hope write to files safely, like databases and version control systems: Leveldb, LMDB, GDBM, HSQLDB, Sqlite, PostgreSQL, Git, Mercurial, HDFS, Zookeeper. They then wrote a static analysis tool that can find incorrect usage of the file API, things like incorrectly assuming that operations that aren't atomic are actually…

And yet all of these systems basically work for day-to-day operations, and fail only under obscure error conditions.

It is totally acceptable for applications to say "I do not support X conditions". Swap out the file half way through a read? Sorry don't support that. Remove power to the storage devise in the middle of a sync operation? Sorry don't support that.

For vital applications, for example databases, this is a known problem and risks of the API are accounted for. Other applications don't have nearly that level of risk associated with them. My music tagging app doesn't need to be resistant to the SSD being struck by lightning.

It is perfectly acceptable to design APIs for 95% of use cases and leave extremely difficult leaks to be solved by the small number of practitioners that really need to solve those leaks.

Re: Working with Files Is Hard (2019)

#72

Post by pjdesno »

wmf wrote: post #53
pjdesno wrote: post #15

Earlier quoted context omitted.

Although the conference this was presented at is platform-agnostic, the author is an expert on Linux, and the motivation for the talk is Linux-specific. (Dropbox dropping support for non-ext4 file systems) The post supports its points with extensive references to prior research - research which hasn't been done in the Microsoft environment. For various reasons (NDAs, etc.) it's likely that no such research will ever…

Universities can get Windows source code under NDA and do research on it but nobody really cares about such work.

"Getting windows source code under NDA" doesn't necessarily mean "can do research on it".

If you can't publish it, it's not research. If the source code is under NDA, then Microsoft gets the final say about whether you can publish or not, and if the result is embarrassing to Microsoft, I'm guessing it's "or not".

Re: Working with Files Is Hard (2019)

#73

Post by jcalvinowens »

huntaub wrote: post #62
IgorPartola wrote: post #61

Earlier quoted context omitted.

Not only that, but the POSIX file API also assumes that NFS is a thing but NFS breaks half the important guarantees of a file system. I don’t know if it’s a baby and bath water situation, but NFS just seems like a whole bunch of problems. It’s like having eval in a programming language.

What aspects of NFS do you think break half of the important guarantees of a file system?

Well, at least O_APPEND, O_EXCL, O_SYNC, and flock() aren't guaranteed to work (although they can with recent versions as I understand it).

UID mapping causing read() to return -EACCES after open() succeeds breaks a lot of userland code.

Re: Working with Files Is Hard (2019)

#74

Post by DiggyJohnson »

jcalvinowens wrote: post #73
huntaub wrote: post #62

Earlier quoted context omitted.

What aspects of NFS do you think break half of the important guarantees of a file system?

Well, at least O_APPEND, O_EXCL, O_SYNC, and flock() aren't guaranteed to work (although they can with recent versions as I understand it). UID mapping causing read() to return -EACCES after open() succeeds breaks a lot of userland code.

Just ran into this one recently trying to replace Docker w/ Podman for a CICD runner. Before anyone protests we have very strong, abnormal requirements on my project preventing most saner architectures. It wasn’t the root cause but the failure behavior was weird due to the behavior you just described.

Re: Working with Files Is Hard (2019)

#77

Post by lmm »

josephg wrote: post #65
ryao wrote: post #39

Earlier quoted context omitted.

Packages consist of multiple files. An atomic file write would not allow packages to be either installed or not installed by APT.

Atomicity could encompass a whole bunch of writes at once. Databases implemented atomic transactions in the 70s. Let’s stop pretending like this is an unsolvable CS problem. Its not.

> Databases implemented atomic transactions in the 70s.

And they have deadlocks as a result, which there is no good easy solution to (generally we work around by having only one program access a given database at a time, and even that is not 100% reliable).

Re: Working with Files Is Hard (2019)

#78

Post by lmm »

yuboyt wrote: post #26
kccqzy wrote: post #17

Earlier quoted context omitted.

By the way, LMDB's main developer Howard Chu responded to the paper. He said, > They report on a single "vulnerability" in LMDB, in which LMDB depends on the atomicity of a single sector 106-byte write for its transaction commit semantics. Their claim is that not all storage devices may guarantee the atomicity of such a write. While I myself filed an ITS on this very topic a year ago, http://www.openldap.org/its/inde…

This assumption was wrong for Intel Optane memory. Power loss could cut the data stream anywhere in the middle. (Note: the DIMM nonvolatile memory version)

Really? A 512-byte sector could get partially written? Did anyone actually observe this, or was it just a case of Intel CYA saying they didn't guarantee anything?

Re: Working with Files Is Hard (2019)

#79

Post by lmm »

ablob wrote: post #19
nodamage wrote: post #16

Earlier quoted context omitted.

Before becoming too overconfident in SQLite note that Rebello et al. ( https://ramalagappan.github.io/pdfs/papers/cuttlefs.pdf ) tested SQLite (along with Redis, LMDB, LevelDB, and PostgreSQL) using a proxy file system to simulate fsync errors and found that none of them handled all failure conditions safely. In practice I believe I've seen SQLite databases corrupted due to what I suspect are two main causes: 1. The…

I believe it is impossible to prevent dataloss if the device powers off during a write. The point about corruption still stands and appears to be used correctly from what I skimmed in the paper. Nice reference.

> I believe it is impossible to prevent dataloss if the device powers off during a write.

Most devices write sectors atomically, and so you can build a system on top of that that does not lose committed data. (Of course if the device powers off during a write then you can lose the uncommitted data you were trying to write, but the point is you don't ever have corruption, you get either the data that was there before the write attempt or the data that is there after).

Re: Working with Files Is Hard (2019)

#80

Post by tliltocatl »

namibj wrote: post #47
x1f604 wrote: post #44

Earlier quoted context omitted.

It is not rare for flash storage devices to lose data on power loss, even data that is FLUSH'd. See https://news.ycombinator.com/item?id=38371307 There are known cases where power loss during a write can corrupt previously written data (data at rest). This is not some rare occurrence. This is why enterprise flash storage devices have power loss protection. See also: https://serverfault.com/questions/923971/is-there-a…

I wish someone would sell an SSD that was at most a firmware update away between regular NVMe drive and ZNS NVMe drive. The latter just doesn't leave much room for the firmware to be clever and just swallow data. Maybe also add a pSLC formatting mode for a namespace so one can be explicit about that capability... It just has to be a drive that's useable as a generic gaming SSD so people can just buy it and have casua…

Unfortunately manifacturers almost always prefer price gouging on features that "CuStOmErS aRe NoT GoInG tO nEeD". Is it even a ZNS device available for someone who isn't a hyperscale datacenter operator nowadays?
Post reply on HN