Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Working with Files Is Hard (2019)

danluu.com

Post reply on HN
Print view View original

81–90 of 123 posts

Re: Working with Files Is Hard (2019)

#81

Post by yuboyt »

lmm wrote: post #78
yuboyt wrote: post #26

Earlier quoted context omitted.

This assumption was wrong for Intel Optane memory. Power loss could cut the data stream anywhere in the middle. (Note: the DIMM nonvolatile memory version)

Really? A 512-byte sector could get partially written? Did anyone actually observe this, or was it just a case of Intel CYA saying they didn't guarantee anything?

Yes, really. "Crash-consistent data structures were proposed by enforcing cacheline-level failure-atomicity" see references in: https://doi.org/10.1145/3492321.3519556

Re: Working with Files Is Hard (2019)

#82

Post by pjc50 »

yahayahya wrote: post #11
wruza wrote: post #3

No mention on ntfs and windows keywords in the article, for those interested.

Is that because the windows APIs are better? Or because businesses build their embedded systems/servers with Windows?

The windows APIs are certainly slower. Apart from IOCP I don't think they're that much different? Oh, and mandatory locking on executable images which are loaded, which has .. advantages and disadvantages (it's why Windows keeps demanding restarts)

Re: Working with Files Is Hard (2019)

#83

Post by Dylan16807 »

count wrote: post #29
SoftTalker wrote: post #21

Earlier quoted context omitted.

Only way I know of is if you have e.g. a RAID controller with a battery-backed write cache. Even that may not be 100% reliable but it's the closest I know of. Of course that's not a software solution at all.

That's uh, not running out of power in the middle of the write. That's having extra special backup power to finish the write. If your battery dies mid cache-write-out, you're still screwed.

I file that under hardware failure, not mundane power loss.

Re: Working with Files Is Hard (2019)

#84

Post by rwmj »

jcalvinowens wrote: post #73
huntaub wrote: post #62

Earlier quoted context omitted.

What aspects of NFS do you think break half of the important guarantees of a file system?

Well, at least O_APPEND, O_EXCL, O_SYNC, and flock() aren't guaranteed to work (although they can with recent versions as I understand it). UID mapping causing read() to return -EACCES after open() succeeds breaks a lot of userland code.

Lack of inotify support is one that has annoyed me in the past. It not only breaks some desktop software, but it also should be possible for NFS to support (after all, the server sees the changes and could notify clients).

Re: Working with Files Is Hard (2019)

#85

Post by kragen »

josephg wrote: post #30
kragen wrote: post #22

Earlier quoted context omitted.

It's not hard to design a less bug-prone API that would enable you to do everything the POSIX file API permits and admits equally-high-performance implementations. But making that new API a replacement for the POSIX API would require rewriting essentially all of the software that somebody cares about to use your new, better API instead of the POSIX API. This is probably only feasible in practice for small embedded sy…

You could do a phased transition, where both the legacy posix api and the new api are available. This has already happened with a lot of the old C standard library. Old, unsafe functions like strcpy were gradually replaced by safer alternatives like strncpy. Database developers don’t want the complexity or poor performance of posix. It’s wild to me that we still don’t have any alternative to fsync in Linux that can a…

There are two serious factual errors in your comment:

- This has not already happened with a lot of the old C standard library. The only function that has ever been removed from the C standard library, to my knowledge, is gets(). In particular, strcpy() has not been removed. Current popular compilers still support gets() with the right options, so it hasn't been removed from the actual library, just the standard.

- strncpy() is not a suitable replacement for strcpy(), certainly not a safer one. It can produce strings missing the terminating null, and it can be slower by orders of magnitude. This has been true since it was introduced in the 01970s. Nearly every call to strncpy() is a bug, and in many cases an exploitable security hole. You are propagating dangerous misinformation. (This is a sign of how difficult it is to make these transitions.)

You also seem to imply that Linux cannot add system calls that are not specified in POSIX, but of course it can and does; openat() and the other 12 related functions, epoll_*(), io_uring_*(), futex_*(), kexec_load(), add_key(), and many others are Linux-specific. The reason barrier() hasn't been added is evidently that the kernel developers haven't been convinced it's worthwhile in the 15+ years since it was proposed, not that POSIX ties their hands.

The nearest equivalents in C for the kind of "staged transition" you are proposing might be things like the 16-bit near/far/huge qualifiers and the Win16 and pre-X MacOS programming models. In each of these cases, a large body of pre-existing software was essentially abandoned and replaced by newly written software.

Re: Working with Files Is Hard (2019)

#86

Post by kragen »

namibj wrote: post #45
josephg wrote: post #30

Earlier quoted context omitted.

You could do a phased transition, where both the legacy posix api and the new api are available. This has already happened with a lot of the old C standard library. Old, unsafe functions like strcpy were gradually replaced by safer alternatives like strncpy. Database developers don’t want the complexity or poor performance of posix. It’s wild to me that we still don’t have any alternative to fsync in Linux that can a…

NVMe has no barrier that doesn't flush the pipeline/ringbuffer of IO requests submitted to it :(

The kernel could implement a non-flushing barrier, even if the underlying device doesn't. You could even do it without any barrier support at all from the underlying device, as long as it reliably tells you when each request has completed; you just don't send it any requests from after the barrier until all the requests before the barrier have completed.

Re: Working with Files Is Hard (2019)

#87

Post by eatonphil »

Retr0id wrote: post #5

> they found that every single piece of software they tested except for SQLite in one particular mode had at least one bug. This is why whenever I need to persist any kind of state to disk, SQLite is the first tool I reach for. Filesystem APIs are scary, but SQLite is well-behaved. Of course, it doesn't always make sense to do that, like the dropbox use case.

Do you turn on SQLite checksumming or how do you feel comfortable that data on disk stays keeps integrity?

Re: Working with Files Is Hard (2019)

#88

Post by jheriko »

this whole thing is a story about using outdated stuff in a shitty ecosystem.

its not a real problem for most modern developers.

pwrite? wtf?

not one mention of fopen.

granted some of the fine detail discussion is interesting, but it doesn't make practical sense since about 1990.

Re: Working with Files Is Hard (2019)

#89

Post by josephg »

kragen wrote: post #85
josephg wrote: post #30

Earlier quoted context omitted.

You could do a phased transition, where both the legacy posix api and the new api are available. This has already happened with a lot of the old C standard library. Old, unsafe functions like strcpy were gradually replaced by safer alternatives like strncpy. Database developers don’t want the complexity or poor performance of posix. It’s wild to me that we still don’t have any alternative to fsync in Linux that can a…

There are two serious factual errors in your comment: - This has not already happened with a lot of the old C standard library. The only function that has ever been removed from the C standard library, to my knowledge, is gets(). In particular, strcpy() has not been removed. Current popular compilers still support gets() with the right options, so it hasn't been removed from the actual library, just the standard. - s…

Yeah, I understand that those methods are still available. But their use is heavily discouraged in new software and a lot of validators & sanitisers will warn if your programs use them. Software itself has largely slowly moved to using the newer, safer methods even though the old methods were never taken away.

I don’t understand the reticence of kernel developers to implement a barrier syscall. I know they could do it. And as this article points out, it would dramatically improve database performance for databases which make use of it. Why hasn’t it happened?

Another commenter says NVMe doesn’t support it natively but I bet hardware vendors would add hardware support if Linux supported it and adding barrier support to their hardware would measurably improve the performance of their devices.

Re: Working with Files Is Hard (2019)

#90

Post by Joker_vD »

kccqzy wrote: post #17
continuational wrote: post #4

> Pillai et al., OSDI’14 looked at a bunch of software that writes to files, including things we'd hope write to files safely, like databases and version control systems: Leveldb, LMDB, GDBM, HSQLDB, Sqlite, PostgreSQL, Git, Mercurial, HDFS, Zookeeper. They then wrote a static analysis tool that can find incorrect usage of the file API, things like incorrectly assuming that operations that aren't atomic are actually…

By the way, LMDB's main developer Howard Chu responded to the paper. He said, > They report on a single "vulnerability" in LMDB, in which LMDB depends on the atomicity of a single sector 106-byte write for its transaction commit semantics. Their claim is that not all storage devices may guarantee the atomicity of such a write. While I myself filed an ITS on this very topic a year ago, http://www.openldap.org/its/inde…

> the developer responds by saying they have manually checked the generated assembler to make sure the assembler is correct at the ISA level even though the C code is wrong at the abstract C machine level, and they commit to checking this in the future.

Yeah, sounds about right about quite a lot of the C programmers except for the "they commit to checking this in the future" part. I've responses like "well, don't upgrade your compiler; I'm gonna put 'Clang >= 9.0 is unsupported' in the README as a fix".

Post reply on HN