Filesystem devs should aim to make “badly written” app code “just work” (2009)

Do you say "you're holding it wrong", or do you adjust to what people seem to be like?

I mean, it's insoluble if treated as a single binary decision and not contextual.

POSIX, and by extension, the classic 1960s-1980s era UNIX way of doing things just needs die a long overdue death.

This stuff was designed at a time when every CPU instruction mattered, everything was optimised to death for frugality, and commands were abbreviated from "copy" to "cp" because ermahgerd two bytes is a huge saving! That mentality got us Y2K. This is an era where latencies were not the bottleneck, CPU cycles and memory bytes were.

A lot of stuff in filesystems is just plain stupid. For example, why do applications install their files. one. at. a. time? Like... what the fuck? How does it make any sense for an application to be partially installed? Who actually codes their application with 500 modules and dynamic libraries to be able to handle the scenario where one of them is inaccessible due to an ACL or a mismatched version because of an overwrite by something or someone else? NOBODY, that's who. Meanwhile, I can make a cup of tea while Adobe Lightroom launches on an SSD drive because it is 99% OS API overhead and 1% usermode action.

This is why Docker is popular. Not because Docker is good, but because OS APIs are retarded.

Every application install should be a union fs. This union fs should be entirely user-mode, so that if an application has 10,000 files, it doesn't take 10,000 round-trips to the OS kernel with the Intel mitigations, context switches, and cache flushes that all brings with it.

Copying a file shouldn't require a user-mode buffer to feed the data through, forcing it to come down WAN links just to go back up the same WAN link again on the way out.

Overwriting a file shouldn't require more than a single API call, because it's nearly 2020, and we should have long since realised that kernel transitions are expensive, so we should optimise to minimise the number of round-trips. open(), write(), write(), write(), flush(), sync(), close(), poke(), prod(), jesusfuckingchrist(). Just take a buffer bigger than 4KB, or better yet, standardise an API to take a stream from user mode.

Just take a buffer and a filename, and atomically replace. Done. Bang. No lost data, not torn writes, just DO IT. How hard can this be? Is it impossible to do this? Are we forever stuck with POSIX, which was created in 1988, before most of its modern users were born?

Linus makes the point here is that the file system API is already complicated to the point that few use or implement it correctly. Further complicating the API will likely create more problems than it fixes.

You're mixing simplicity of API with simplicity of implementation. More often than not, you can only have one but not both.

Modern Linux or Windows do huge amount of magic when you call kernel API like open (POSIX) / CreateFile (Windows), yet the API is simple and easy.

You can expose all implementation details, your code will be simple, but hard to build upon. Speaking about data storage, once upon a time I programmed Nintendo consoles, their file system API probably was very simple for Nintendo to implement, but using it wasn't fun: SDK documentation specified delays, specified how to deal with corrupt flash memory, etc.

You can do the other way, you'll have to do lot of work handling all the edge cases, your code will be very complex, but this way you might make a system that's actually useful. About data storage, SQL servers have tons of internal complexity, even sqlight does, but API, the SQL, is high level and easy to use even by non-programmers.