Files Are Fraught with Peril • Hacker News Board

Re: Files Are Fraught with Peril

#11

Post by yorwba » Tue, Jul 23, 2019, 9:33 AM UTC

bjhkx wrote: post #9

ChrisSD wrote: post #6

Earlier quoted context omitted.

ext4 is by far the most popular filesystem so supporting it is a necessity for that reason alone. Nothing to do with it being superior or special. The argument this article is making is that supporting additional filesystems is hard. The is meant to refute the allegation that it's trivial to add whatever other filesystems the OS supports.

No, that's the argument this article is supposedly making. But it's not actually making it. That's the problem. Filesystems are an abstraction and I expected to see some problems regarding a leaky abstraction or something, but the article doesn't mention anything like that. It mentions that dealing with hard disks is hard (which I believe, since they are flaky hardware). But dropbox didn't say "we won't be supporting…

From the article: "Large parts of the file API look like this, where behavior varies across filesystems or across different modes of the same filesystem. For example, if we look at mainstream filesystems, appends are atomic, except when using ext3 or ext4 with data=writeback, or ext2 in any mode and directory operations can't be re-ordered w.r.t. any other operations, except on btrfs."

That doesn't mean ext4 is problem-free, but it does mean that other filesystems have different problems that are not fixed by mitigations for ext4's quirks.

Re: Files Are Fraught with Peril

#12

Post by bjhkx » Tue, Jul 23, 2019, 9:39 AM UTC

yorwba wrote: post #11

bjhkx wrote: post #9

Earlier quoted context omitted.

No, that's the argument this article is supposedly making. But it's not actually making it. That's the problem. Filesystems are an abstraction and I expected to see some problems regarding a leaky abstraction or something, but the article doesn't mention anything like that. It mentions that dealing with hard disks is hard (which I believe, since they are flaky hardware). But dropbox didn't say "we won't be supporting…

From the article: "Large parts of the file API look like this, where behavior varies across filesystems or across different modes of the same filesystem. For example, if we look at mainstream filesystems, appends are atomic, except when using ext3 or ext4 with data=writeback, or ext2 in any mode and directory operations can't be re-ordered w.r.t. any other operations, except on btrfs." That doesn't mean ext4 is probl…

Okay and? Why does userspace care about this? Should Emacs not run on btrfs?

Re: Files Are Fraught with Peril

#13

Post by ChrisSD » Tue, Jul 23, 2019, 9:51 AM UTC

greatgib wrote: post #8

I did work on backup solutions and supported Dropbox, and I think that the author used Dropbox as a reference to support his own concerns about filesystems that are not related in any case to the Dropbox case. From the filesystem point of view, I don't think that Dropbox is so much concerned about you losing your data because of corruption. As, anyway they are supposed to be safe and versioned in the cloud. And I thi…

Yeah, the article used comments on Dropbox as a kicking off point to talk more generally about filesystem issues. The author is aware that in this specific case it was more about metadata:

> I believe their hand was forced by the way they want to store/use data, which they can only do with ext, but even if that wasn't the case...

The Dropbox examples seem to have clouded the issues actually being discussed.

Re: Files Are Fraught with Peril

#14

Post by alkonaut » Tue, Jul 23, 2019, 10:01 AM UTC

pjc50 wrote: post #10

alkonaut wrote: post #5

With the risk of sounding like those /r/programming replies: why would an application be worried about journaling/logs on a file system level? As an application developer all I’m usually told is that the only atomic operations are creates, deletes, and renames. So to update a file you always write a second file and then rename it to the destination. So Copy file.txt to file.txt.new Update file.txt.new Rename file.txt…

This is specifically addressed at one point: > This trick doesn't work. People seem to think that this is safe becaus the POSIX spec says that rename is atomic, but that only means rename is atomic with respect to normal operation, that doesn't mean it's atomic on crash. This isn't just a theoretical problem; if we look at mainstream Linux filesystems, most have at least one mode where rename isn't atomic on crash. R…

Wow I can’t believe I missed the section on this. My bad. Rename not being atomic on crash sounds absolutely terrifying. Luckily for NTFS there is an atomic rename (deprecated but still working).

That said, what kind of non-atomicness occurs is also important. Leaving both the source and destination files around is at least something that can be recovered from, but if there are other failure modes that would be scary!

This is how nearly all “document-based” applications prevent corruption also under posix, because nearly no applications would use per-filesystem logic. I suppose ”prevent corruption with a crash at any point” is simply too high a bar for most apps - they would resort to backups to prevent data loss with untimely crashes?

With networks- or removable media I know all sorts of madness will happen.

Re: Files Are Fraught with Peril

#15

Post by auggierose » Tue, Jul 23, 2019, 10:33 AM UTC

iOS 13 changes a lot about how files work, also internally, so implementing a correct File Provider is going to be quite hard (even the iCloud one doesn't work properly yet in the beta ...). I hope Dropbox and the others follow suit and implement this correctly. Just another example of how files are difficult to get right.

Re: Files Are Fraught with Peril

#16

Post by pjc50 » Tue, Jul 23, 2019, 10:35 AM UTC

bjhkx wrote: post #12

yorwba wrote: post #11

Earlier quoted context omitted.

From the article: "Large parts of the file API look like this, where behavior varies across filesystems or across different modes of the same filesystem. For example, if we look at mainstream filesystems, appends are atomic, except when using ext3 or ext4 with data=writeback, or ext2 in any mode and directory operations can't be re-ordered w.r.t. any other operations, except on btrfs." That doesn't mean ext4 is probl…

Okay and? Why does userspace care about this? Should Emacs not run on btrfs?

This is rather like the bluescreen problem: if your application tries to open a file from the normal filesystem and it's corrupt, the user blames the application. If the user opens a file in the Dropbox folder, they blame Dropbox. So they end up engaging in heroics to not be blamed for it.

(Windows has gone to increasing lengths to accomodate and contain badly written drivers, since most bluescreens are caused by drivers. There is now a subsystem to allow the video drivers to crash and entirely restart without bluescreening.)

Re: Files Are Fraught with Peril

#17

Post by pjc50 » Tue, Jul 23, 2019, 10:38 AM UTC

ovi256 wrote: post #4

This is fascinating, a bit in the same way that looking at accidents is interesting. A good synthesis is: filesystem API design is obviously a problem, given that people that specialize in using them can't do it correctly: "Pillai et al., OSDI’14 looked at a bunch of software that writes to files, including things we'd hope would write to files safely, like datbases and version control systems ... they found that eve…

> filesystem API design is obviously a problem

It's an API that's so critical it's almost impossible to change or rewrite. The API itself has barely moved in 30 years, whether it's POSIX or Win32. Possibly the only widely adopted change in filesystem API has been "S3" and compatibles, which provide a completely different set of atomicity semantics as well as being network-native.

Re: Files Are Fraught with Peril

#18

Post by networkimprov » Tue, Jul 23, 2019, 10:59 AM UTC

The suggestion to "use SQLite" to write files should come with a caveat. Such files can still become corrupted, and cannot be inspected nor amended with a text editor.

Any truly robust file-on-local-disk storage scheme for a non-cloud system should allow for manual diagnostics and repair.

Re: Files Are Fraught with Peril

#19

Post by harperlee » Tue, Jul 23, 2019, 11:03 AM UTC

networkimprov wrote: post #18

The suggestion to "use SQLite" to write files should come with a caveat. Such files can still become corrupted, and cannot be inspected nor amended with a text editor. Any truly robust file-on-local-disk storage scheme for a non-cloud system should allow for manual diagnostics and repair.

Text-based formats are the best for repairability but not so efficient for structuring, querying and storing information. It’s a tradeoff, as always!

Re: Files Are Fraught with Peril

#20

Post by cjfd » Tue, Jul 23, 2019, 11:13 AM UTC

networkimprov wrote: post #18

The suggestion to "use SQLite" to write files should come with a caveat. Such files can still become corrupted, and cannot be inspected nor amended with a text editor. Any truly robust file-on-local-disk storage scheme for a non-cloud system should allow for manual diagnostics and repair.

Also, schema updates are quite hopeless with sqlite. One cannot even rename a column last time I checked.