Live data from Hacker News

Files Are Fraught with Peril

danluu.com

71–79 of 79 posts

Re: Files Are Fraught with Peril

#71
post #52

Earlier quoted context omitted.

I would only add that mmap should be used with care[0] and pread should be preferred. [0]: https://www.sublimetext.com/blog/articles/use-mmap-with-care

It should be noted that much of the problems with mmap cited there aren't actually mmap's fault, but the fact that I/O errors become POSIX signals, and the API for POSIX signals really blows. (POSIX signals are probably even more ripe than filesystems for needing a different approach). I'm surprised that page didn't mention the other issue with mmap, which is concurrent file access.

How would you signal errors accessing virtual memory?

In the context of a VM error, the only deficiency with POSIX signals I can think of off-hand is that POSIX only permits global signal handlers--not per thread--but that's only an issue for the multithreaded case.

Re: Files Are Fraught with Peril

#72
post #41

I appreciate the desire for stripped-down websites, but "body { max-width: 800px; }" makes a world of difference for readability.

This is where em rather than px (or rem) units is preferred.

   body {
      margin: 2rem auto;
      padding: 2rem;
      width: auto;
      max-width:  45em;
    }
Works in most instances, even without @media queries.

https://codepen.io/dredmorbius/full/KpMqqB

Re: Files Are Fraught with Peril

#73

Where can I learn more about the rename() trick not being safe? I was also (incorrectly) assuming that renaming a file after having fsynced the contents is a valid mechanism to perform "atomic" file writes where the write either appears completely at the destination path, creates a zero byte file or does not happen at all. But it should never produce an "incorrect" non-zero byte file - even in the face of crashes. It…

We wrote a program where atomicity was mission critical, and tested the hell out of power loss recovery. I can confirm experimentally that the rename trick is not good enough.

It was nearly good enough on Linux. Corruption was very rare, but we still caught issues. On Windows corruption happened something like 1 in 10 poweroffs.

For low performance ACID we write the whole file out twice and fsync between each write. First write is to the backup file, second is to the primary file. This method passed our tests.

For high performance, we use another strategy entirely that's a lot more involved.

Re: Files Are Fraught with Peril

#74
post #35
post #23

Does anyone happen to know if ZFS also suffers from renames being non-atomic? Since ZFS is the only candidate for a reasonable file system we have anyway[+], I'd be a lot less sad if it turns out that way out of this dumpster fire is just tell people to use ZFS. [+] The absolutely minimal requirement for a reasonable file system is that it works on multiple popular platforms and performs checksumming and other data i…

I love ZFS but the licence situation means it'll always be a mess to use.

Yes, it's sad. Even more so since there is not even a potential competitor in sight.

Re: Files Are Fraught with Peril

#75
post #71

Earlier quoted context omitted.

It should be noted that much of the problems with mmap cited there aren't actually mmap's fault, but the fact that I/O errors become POSIX signals, and the API for POSIX signals really blows. (POSIX signals are probably even more ripe than filesystems for needing a different approach). I'm surprised that page didn't mention the other issue with mmap, which is concurrent file access.

How would you signal errors accessing virtual memory? In the context of a VM error, the only deficiency with POSIX signals I can think of off-hand is that POSIX only permits global signal handlers--not per thread--but that's only an issue for the multithreaded case.

I'm partial to something more akin to SEH for the synchronous signals (SIGILL, SIGFPE, SIGSEGV, SIGBUS, SIGTRAP). Basically, define that synchronous signals are handled per-thread in a manner similar to try/catch, although you need an extra catch type that amounts to "retry the operation" in addition to "rethrow" and "swallow the exception".

Re: Files Are Fraught with Peril

#76
post #71

Earlier quoted context omitted.

How would you signal errors accessing virtual memory? In the context of a VM error, the only deficiency with POSIX signals I can think of off-hand is that POSIX only permits global signal handlers--not per thread--but that's only an issue for the multithreaded case.

I'm partial to something more akin to SEH for the synchronous signals (SIGILL, SIGFPE, SIGSEGV, SIGBUS, SIGTRAP). Basically, define that synchronous signals are handled per-thread in a manner similar to try/catch, although you need an extra catch type that amounts to "retry the operation" in addition to "rethrow" and "swallow the exception".

Can you restart execution at the point of the fault using SEH? A brief skim of the documentation doesn't seem to suggest it's possible as a general matter. If I wanted to implement a dynamically growable stack structure in a way that let me resume at the point of the fault, preserving program state, how would I do that?

I ask because I think sometimes people conflate POSIX signals offering poor semantics (i.e. reentrancy issues) with POSIX signals being too low-level. I can imagine how I might implement SEH using POSIX signals (though a per-thread handler would be really nice), but not vice-versa (though maybe it is possible).

As I understand it, there's a long history behind signals relating to interrupt vs polling software system models. Signals as they exist in Unix were a very early implementation of the interrupt driven model in the context of a kerneluser space interface. But Unix's process and I/O concepts were perhaps too convenient so the value-add of the signals model was minimal; Unix ended up evolving in directions that didn't require the interrupt abstraction (at least, not until decades later). This history explains why POSIX signals are so low-level and the lack of comprehensive runtime treatment.

Note that they're only low-level by today's standards. At the time they were very high-level--a signal interrupt magically preserved your process state (stack and program counter) and would magically resume the process when returning from the handler, which could be a simple C function. Even better, this could occur recursively! And it's worth mentioning that all kernel interfaces were and remain async-signal safe (e.g. dup2 is atomic even from the perspective of an async signal).[1] The lack of consistent and convenient treatment by the runtime comes from the fact that much of the runtime we're most familiar with came later; threads came way later. When they came about people had already moved away from signals, perhaps because they saw that it was too much work to make the interrupt driven model work well at a high-level.

[1] In classic Unix style it did all this with the most minimal of kernel and user space code, pushing process state onto the user space stack and relying on an in-process trampoline to restore program state (which is how recursion could be supported without any complexity in kernel space).

Re: Files Are Fraught with Peril

#77
post #70

Earlier quoted context omitted.

ZFS never reorders metadata operations in an observable manner, so it's really well behaved. If only every filesystem was like that... You still need fsync for the data, but you can set sync=disabled on the filesystem, which turns it into a barrier. Alas, you can't do anything more granular than per-filesystem.

Does this idiom (which turns out to be broken on rename op failure for most FSs) work reliably with ZFS (on the same filesystem)? def atomically_write(filename, data): with open(filename + '.tmp', 'w') as fh: fh.write(data) os.rename(filename + '.tmp', filename) In addition to ordering guarantees, you also need to guarantee that even in case of failure the rename operation leaves the to-be-renamed file as it is and t…

I'm actually not sure if that code would work, but I don't think it would. You need at least one fsync in there, before the rename.

Re: Files Are Fraught with Peril

#78
post #54

Earlier quoted context omitted.

That's where the "and versioned" comes in handy

Versioned wont help if it's the first copy that got into dropbox.

If it's the first copy, it will just be read by Dropbox. There will not be 'write' from Dropbox that can corrupt it...

Re: Files Are Fraught with Peril

#79
post #73

Where can I learn more about the rename() trick not being safe? I was also (incorrectly) assuming that renaming a file after having fsynced the contents is a valid mechanism to perform "atomic" file writes where the write either appears completely at the destination path, creates a zero byte file or does not happen at all. But it should never produce an "incorrect" non-zero byte file - even in the face of crashes. It…

We wrote a program where atomicity was mission critical, and tested the hell out of power loss recovery. I can confirm experimentally that the rename trick is not good enough. It was nearly good enough on Linux. Corruption was very rare, but we still caught issues. On Windows corruption happened something like 1 in 10 poweroffs. For low performance ACID we write the whole file out twice and fsync between each write.…

> For high performance, we use another strategy entirely that's a lot more involved.

It sounds interesting. Is there a write-up with more details to read?

Post reply on HN