Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unix’s file durability problem

utcc.utoronto.ca

Post reply on HN
Print view View original

141–150 of 161 posts

Re: Unix’s file durability problem

#141

Post by Someone »

derefr wrote: post #46
dap wrote: post #43

Earlier quoted context omitted.

How would it be different than the filesystem API? There was a great lightning talk a few years ago that I can't seem to find where the author described an API for storing blobs in a hierarchical namespace. Of course, halfway through, it became clear that it's just the POSIX API: you can "open" handles to objects, "rename" them, remove them, and so on. You'd end a transaction with "fsync()". (Okay, that one's a littl…

Two key differences: 1. "MVCC": the ability to effectively get a handle on a static copy of the entire filesystem , perform mutations to many objects, and then submit the transaction, at which point it will either commit or rollback depending on whether any of those objects have been modified. Windows actually has this: https://en.wikipedia.org/wiki/Transactional_NTFS allows for exactly the sort of MVCC I'm talking a…

It surprised me to read that Microsoft is considering to remove transactional NTFS because "there has been extremely limited developer interest in this API platform since Windows Vista primarily due to its complexity and various nuances which developers need to consider as part of application development" (https://msdn.microsoft.com/en-us/library/windows/desktop/hh8...; linked-to from that Wikipedia page)

Re: Unix’s file durability problem

#142

Post by leni536 »

Animats wrote: post #67
jstimpfle wrote: post #59

Earlier quoted context omitted.

> - "Unit" files commit when closed properly (this does not include a program exit without close), and then replace the old version of the file. The file system should guarantee that, after a crash, you have either the old version or the new complete version. Incidentally, this is the way it currently is. From rename(2) : If "newpath already exists, it will be atomically replaced". Just don't forget that "replacing"…

Rename isn't quite an atomic replacement. If you crash before the rename, the new file hangs around. (Hence unwanted .part files.) O_APPEND isn't airtight on all systems. On some older UNIX systems, multiple writers created with "open()" (not "dup()") do not share a file position. NTFS doesn't do append correctly. How do you guarantee that, after a crash, the end of the file is at the end of some write? By updating t…

> Rename isn't quite an atomic replacement. If you crash before the rename, the new file hangs around. (Hence unwanted .part files.)

It seems O_TMPFILE cannot be used to atomically replace existing files either. If only one could use renameat2() on a raw fd or force linkat() to an atomic overwrite.

Re: Unix’s file durability problem

#143

Post by sdk77 »

brigade wrote: post #140
drewm1980 wrote: post #136

Earlier quoted context omitted.

You have just described a UPS. Or a laptop. A battery just makes more sense than a capacitor in this application.

Why? Wouldn't a large enough capacitor last longer than batteries? UPS batteries I've seen are only rated for 3 years, and half of my recent laptop batteries have physically swelled after 3 years of use.

It would, but the size would be impractical and it's too costly. Quick estimate, let's assume we need 10W for 10 seconds, that's 100 Joule of energy. The energy stored in a capacitor is 0.5CV^2. Say we use a 10V capacitor then C = 2 Farad. They exist, but they are very large (look them up on amazon for instance). You'll probably need more like twice the capacity though because it's impossible to extract all energy from the capacitor, and it's lossy to convert it to a constant +5V / +12V.

Re: Unix’s file durability problem

#144

Post by POSIXprog »

Most programmers, and especially the hipsterish HN crowd, simply cannot be trusted to write correct file manipulation code. In general, anything they produce will be riddled with race conditions and erroneous assumptions (e.g., that rename works cross-device, that close cannot fail) and that breaks in rare but possibly catastrophic circumstances.

The solution is copy-on-write file systems such as ZFS and Btrfs that ensure neither data nor meta-data are ever altered in-place and the reuse of correct file manipulation code (written by adults) rather than rolling your own, either from a library or something higher-level like SQLite.

Re: Unix’s file durability problem

#145

Post by rikkus »

I use something akin to djb's Maildir delivery procedure when I want to be as close to sure as I can be that a file 'has been saved'.

1. Create a temp file on the same mount

2. Write data to file, checking return of each write(), then a final fsync(), and close()

3. link() to filename we actually want

I still haven't figured out how to be this safe on Windows, because AFAICT there's no atomic way to link() or 'move' a file - and they canned the transactional API for the filesystem. Any pointers to how to write files 'safely' on Windows would be much appreciated!

Re: Unix’s file durability problem

#146

Post by the_why_of_y »

jstimpfle wrote: post #38
Ericson2314 wrote: post #24

Yet another example about how the filesystem, and its POSIX realization, is in fact a horrible abstraction.

I don't think there is anything wrong with the POSIX file system API. Two things: - I think it's mainly the modern file systems like Btrfs and I think partly also ext4, which introduced a shift in paradigm, which broke old applications (or at least broke their performance, for example dpkg). - We're talking about a hierarchical file system, meaning it's easy for humans to find data, but terrible for machines because…

It wasn't ext4 that introduced a paradigm shift but ext3: the fsync() implementation in ext3 by design wrote all buffered writes for the whole filesystem, not just a single file, hence it was so horribly slow that it taught a generation of application programmers never to call fsync() to avoid multi-second pauses.

Re: Unix’s file durability problem

#147

Post by x0 »

kentonv wrote: post #79
Animats wrote: post #67

Earlier quoted context omitted.

Rename isn't quite an atomic replacement. If you crash before the rename, the new file hangs around. (Hence unwanted .part files.) O_APPEND isn't airtight on all systems. On some older UNIX systems, multiple writers created with "open()" (not "dup()") do not share a file position. NTFS doesn't do append correctly. How do you guarantee that, after a crash, the end of the file is at the end of some write? By updating t…

> How do you guarantee that, after a crash, the end of the file is at the end of some write? By updating the file size after the write. Note that Linux ext4 does not do this. On a power outage, you can get bogus trailing zeros on a file which you were appending, because the file size was updated before the data was written. I asked Ted T'so about this and he said it was working as intended. https://plus.google.com/+K…

Oh! So that's where all the random nulls in my shell history came from. Zsh would sometimes complain about a corrupted history after a bad crash, and I always wondered what caused maybe 20 nulls to be added to the end of the file. Thanks!

Re: Unix’s file durability problem

#148

Post by avar »

cm2187 wrote: post #131

Stupid question: why don't all computers (irrespective of the OS) have a built in power loss mechanism? It seems to be such a common and obvious problem. 1. The PSU would have a big enough capacitor to keep the computer running for a few seconds at its stated output power 2. The PSU would notify the OS of a power loss 3. The OS would immediately flush all caches and adopt a "brace position". 4. Events are spread syst…

You'll find this old mailing list post a good answer to your question: http://zork.net/~nick/mail/why-reiserfs-is-teh-sukc

The answer is cost. We used to have the computers you're describing. They were made by SGI and the hardware & OS made all sorts of guarantees like the ones you're describing, to the point where XFS was ported to Linux x86 initially it had all sorts of bugs that simply couldn't happen on SGI machines.

The x86 boxes were cheaper, nobody really cared enough about hardware reliability enough to pay the price for the likes of SGI, and the rest is history. Today we have a "worse is better" hardware architecture and software has to be able to handle it.

Re: Unix’s file durability problem

#149

Post by cm2187 »

avar wrote: post #148
cm2187 wrote: post #131

Stupid question: why don't all computers (irrespective of the OS) have a built in power loss mechanism? It seems to be such a common and obvious problem. 1. The PSU would have a big enough capacitor to keep the computer running for a few seconds at its stated output power 2. The PSU would notify the OS of a power loss 3. The OS would immediately flush all caches and adopt a "brace position". 4. Events are spread syst…

You'll find this old mailing list post a good answer to your question: http://zork.net/~nick/mail/why-reiserfs-is-teh-sukc The answer is cost. We used to have the computers you're describing. They were made by SGI and the hardware & OS made all sorts of guarantees like the ones you're describing, to the point where XFS was ported to Linux x86 initially it had all sorts of bugs that simply couldn't happen on SGI machi…

Very interesting link. Thanks

Re: Unix’s file durability problem

#150

Post by jstimpfle »

dspeyer wrote: post #125
jstimpfle wrote: post #59

Earlier quoted context omitted.

> - "Unit" files commit when closed properly (this does not include a program exit without close), and then replace the old version of the file. The file system should guarantee that, after a crash, you have either the old version or the new complete version. Incidentally, this is the way it currently is. From rename(2) : If "newpath already exists, it will be atomically replaced". Just don't forget that "replacing"…

Rename doesn't always do what we want. The one serious investigation I've seen [1] unhelpfully states that "[W]hen a file is appended, and then renamed ... many file systems recognize it and allocate blocks immediately." Suggesting that some file systems don't, but failing to name them. In general, appends followed by directory operations are not order-preserved. [1] https://www.usenix.org/system/files/conference/osd…

There's no order preserving guarantee. Code should do

    fd
(Strictly speaking there is a problem in the rename because we can't be sure "tmpfile" still refers to the same inode. It would be nice to have a way to link fds, but I'm not aware of any).

According to the paper, existing file systems recognize the (open,) write, rename pattern and don't reorder write and rename. That would break too much existing code. But these are filesystem-specific design decisions.

Btw. it's just the same with CPUs writing to memory. Generally writes can be reordered if the behaviour is not visible to the executing thread. However if there are multiple threads you might need to insert (CPU-specific) memory barriers to prevent reorderings that are visible to other threads. x86 is like those file systems here in that most incorrect code still works, because x86 makes some ordering guarantees.

Post reply on HN