Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Working with Files Is Hard (2019)

danluu.com

Post reply on HN
Print view View original

21–30 of 123 posts

Re: Working with Files Is Hard (2019)

#21

Post by SoftTalker »

ablob wrote: post #19
nodamage wrote: post #16

Earlier quoted context omitted.

Before becoming too overconfident in SQLite note that Rebello et al. ( https://ramalagappan.github.io/pdfs/papers/cuttlefs.pdf ) tested SQLite (along with Redis, LMDB, LevelDB, and PostgreSQL) using a proxy file system to simulate fsync errors and found that none of them handled all failure conditions safely. In practice I believe I've seen SQLite databases corrupted due to what I suspect are two main causes: 1. The…

I believe it is impossible to prevent dataloss if the device powers off during a write. The point about corruption still stands and appears to be used correctly from what I skimmed in the paper. Nice reference.

Only way I know of is if you have e.g. a RAID controller with a battery-backed write cache. Even that may not be 100% reliable but it's the closest I know of. Of course that's not a software solution at all.

Re: Working with Files Is Hard (2019)

#22

Post by kragen »

jcranmer wrote: post #18
__loam wrote: post #10

Earlier quoted context omitted.

POSIX is also so old and essential that it's hard to imagine an alternative.

Not really, there's been lots of APIs that have improved on the POSIX model. The kind of model I prefer is something based on atomicity. Most applications can get by with file-level atomicity--make whole file read/writes atomic with a copy-on-write model, and you can eliminate whole classes of filesystem bugs pretty quickly. (Note that something like writeFileAtomic is already a common primitive in many high-level fi…

It's not hard to design a less bug-prone API that would enable you to do everything the POSIX file API permits and admits equally-high-performance implementations. But making that new API a replacement for the POSIX API would require rewriting essentially all of the software that somebody cares about to use your new, better API instead of the POSIX API. This is probably only feasible in practice for small embedded systems with a fairly small universe of software.

Re: Working with Files Is Hard (2019)

#23

Post by wruza »

yahayahya wrote: post #11
wruza wrote: post #3

No mention on ntfs and windows keywords in the article, for those interested.

Is that because the windows APIs are better? Or because businesses build their embedded systems/servers with Windows?

I doubt that, was just curious how it might compare in the article.

Re: Working with Files Is Hard (2019)

#24

Post by timewizard »

Retr0id wrote: post #9
continuational wrote: post #4

> Pillai et al., OSDI’14 looked at a bunch of software that writes to files, including things we'd hope write to files safely, like databases and version control systems: Leveldb, LMDB, GDBM, HSQLDB, Sqlite, PostgreSQL, Git, Mercurial, HDFS, Zookeeper. They then wrote a static analysis tool that can find incorrect usage of the file API, things like incorrectly assuming that operations that aren't atomic are actually…

> why the file API so hard to use that even experts make mistakes? I think the short answer is that the APIs are bad. The POSIX fs APIs and associated semantics are so deeply entrenched in the software ecosystem (both at the OS level, and at the application level) that it's hard to move away from them.

> POSIX fs APIs and associated semantics

Well I think that's the actual problem. POSIX gives you an abstract interface but it essentially does not enforce any particular semantics on those interfaces.

Re: Working with Files Is Hard (2019)

#25

Post by timewizard »

jcranmer wrote: post #18
__loam wrote: post #10

Earlier quoted context omitted.

POSIX is also so old and essential that it's hard to imagine an alternative.

Not really, there's been lots of APIs that have improved on the POSIX model. The kind of model I prefer is something based on atomicity. Most applications can get by with file-level atomicity--make whole file read/writes atomic with a copy-on-write model, and you can eliminate whole classes of filesystem bugs pretty quickly. (Note that something like writeFileAtomic is already a common primitive in many high-level fi…

> make whole file read/writes atomic with a copy-on-write model,

I have many files that are several GB. Are you sure this is a good idea? What if my application only requires best effort?

> eliminate whole classes of filesystem bugs pretty quickly.

Block level deduplication is notoriously difficult.

> where the only kind of write allowed is to atomically append a chunk of data to the file

Which sounds good until you think about the complications involved in block oriented storage medium. You're stuck with RMW whether you think you're strictly appending or not.

Re: Working with Files Is Hard (2019)

#26

Post by yuboyt »

kccqzy wrote: post #17
continuational wrote: post #4

> Pillai et al., OSDI’14 looked at a bunch of software that writes to files, including things we'd hope write to files safely, like databases and version control systems: Leveldb, LMDB, GDBM, HSQLDB, Sqlite, PostgreSQL, Git, Mercurial, HDFS, Zookeeper. They then wrote a static analysis tool that can find incorrect usage of the file API, things like incorrectly assuming that operations that aren't atomic are actually…

By the way, LMDB's main developer Howard Chu responded to the paper. He said, > They report on a single "vulnerability" in LMDB, in which LMDB depends on the atomicity of a single sector 106-byte write for its transaction commit semantics. Their claim is that not all storage devices may guarantee the atomicity of such a write. While I myself filed an ITS on this very topic a year ago, http://www.openldap.org/its/inde…

This assumption was wrong for Intel Optane memory. Power loss could cut the data stream anywhere in the middle. (Note: the DIMM nonvolatile memory version)

Re: Working with Files Is Hard (2019)

#27

Post by thfuran »

dkarl wrote: post #12
continuational wrote: post #4

> Pillai et al., OSDI’14 looked at a bunch of software that writes to files, including things we'd hope write to files safely, like databases and version control systems: Leveldb, LMDB, GDBM, HSQLDB, Sqlite, PostgreSQL, Git, Mercurial, HDFS, Zookeeper. They then wrote a static analysis tool that can find incorrect usage of the file API, things like incorrectly assuming that operations that aren't atomic are actually…

> why the file API so hard to use that even experts make mistakes? Sounds like Worse Is Better™: operating systems that tried to present safer abstractions were at a disadvantage compared to operating systems that shipped whatever was easiest to implement. (I'm not an expert in the history, just observing the surface similarity and hoping someone with more knowledge can substantiate it.)

Why does that seem more likely than file system API simply not having been a major factor in the success of failure of OSes?

Re: Working with Files Is Hard (2019)

#28

Post by userbinator »

I don't get it. The only times I've had problems with filesystem corruption in the past few decades was with a hardware problem, and said hardware was quickly replaced. FAT family has been perfectly fine while I've encountered corruption on every other FS including NTFS, exFAT, and the ext* family.

Meanwhile you can read plenty of stories of others having the exact opposite experience.

If you keep losing data to power losses or crashes, perhaps fix the cause of that? It doesn't make sense to try to work around it.

Re: Working with Files Is Hard (2019)

#29

Post by count »

SoftTalker wrote: post #21
ablob wrote: post #19

Earlier quoted context omitted.

I believe it is impossible to prevent dataloss if the device powers off during a write. The point about corruption still stands and appears to be used correctly from what I skimmed in the paper. Nice reference.

Only way I know of is if you have e.g. a RAID controller with a battery-backed write cache. Even that may not be 100% reliable but it's the closest I know of. Of course that's not a software solution at all.

That's uh, not running out of power in the middle of the write. That's having extra special backup power to finish the write. If your battery dies mid cache-write-out, you're still screwed.

Re: Working with Files Is Hard (2019)

#30

Post by josephg »

kragen wrote: post #22
jcranmer wrote: post #18

Earlier quoted context omitted.

Not really, there's been lots of APIs that have improved on the POSIX model. The kind of model I prefer is something based on atomicity. Most applications can get by with file-level atomicity--make whole file read/writes atomic with a copy-on-write model, and you can eliminate whole classes of filesystem bugs pretty quickly. (Note that something like writeFileAtomic is already a common primitive in many high-level fi…

It's not hard to design a less bug-prone API that would enable you to do everything the POSIX file API permits and admits equally-high-performance implementations. But making that new API a replacement for the POSIX API would require rewriting essentially all of the software that somebody cares about to use your new, better API instead of the POSIX API. This is probably only feasible in practice for small embedded sy…

You could do a phased transition, where both the legacy posix api and the new api are available. This has already happened with a lot of the old C standard library. Old, unsafe functions like strcpy were gradually replaced by safer alternatives like strncpy.

Database developers don’t want the complexity or poor performance of posix. It’s wild to me that we still don’t have any alternative to fsync in Linux that can act as a barrier without also flushing caches at the same time.

Post reply on HN