Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Filesystem devs should aim to make “badly written” app code “just work” (2009)

lwn.net

Post reply on HN
Print view View original

91–100 of 117 posts

Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)

#91

Post by heavenlyblue »

jiggawatts wrote: post #43
bvinc wrote: post #29

I think he's got a point. As a developer I find myself in a different scenario. I'm usually trying to find out what exactly the 100% guaranteed way to do something is. Instead, I find incomplete documentation and different people with different opinions on what the guarantees are, and most people writing bad code that they assume will usually work. Just modifying a file in an atomic way requires a complicated dance o…

I do, but it's not a popular opinion. POSIX, and by extension, the classic 1960s-1980s era UNIX way of doing things just needs die a long overdue death. This stuff was designed at a time when every CPU instruction mattered, everything was optimised to death for frugality, and commands were abbreviated from "copy" to "cp" because ermahgerd two bytes is a huge saving! That mentality got us Y2K. This is an era where lat…

All of what you are describing is already a product on top of the APIs you are describing?

Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)

#92

Post by pas »

jiggawatts wrote: post #43
bvinc wrote: post #29

I think he's got a point. As a developer I find myself in a different scenario. I'm usually trying to find out what exactly the 100% guaranteed way to do something is. Instead, I find incomplete documentation and different people with different opinions on what the guarantees are, and most people writing bad code that they assume will usually work. Just modifying a file in an atomic way requires a complicated dance o…

I do, but it's not a popular opinion. POSIX, and by extension, the classic 1960s-1980s era UNIX way of doing things just needs die a long overdue death. This stuff was designed at a time when every CPU instruction mattered, everything was optimised to death for frugality, and commands were abbreviated from "copy" to "cp" because ermahgerd two bytes is a huge saving! That mentality got us Y2K. This is an era where lat…

This already exists. O_DIRECT exists, now there's io_urig too (plus the libaio1 which uses io_submit the old async API).

The problem is not with files. Kernel transitions are expensive, but the problem is not that. It's metadata sync (file inode + directory inode) and so on.

Install is slow on Windows because braindeadness of vendors (cygwin is fast to install). Package stuff is slow on Linux, because apt calls out to dpkg, which first reads the package db. Which is a plain text file with newlines as separators and it has to be parsed. Yum is slow because by default it syncs repos on install, etc.

Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)

#93

Post by TeMPOraL »

silvestrov wrote: post #86
Animats wrote: post #59

I'd argue that UNIX-type file systems should offer several types of files: * Unit files. When you create a file and write it, it's not visible for other opens until you close it. If you open a file with O_CREAT|O_WRONLY|O_TRUNC, you create a new file, which replaces the old one on close. In the event of a program or system crash, or exiting via "abort" without closing first, the old file remains. So there's always on…

> Unit files. ... you create a new file, which replaces the old one on close. Classic MacOS-9 had the PBExchangeFiles call which did this perfectly. Before call: dirEntryA -> fileContentsA dirEntryB -> fileContentsB after call dirEntryA -> fileContentsB dirEntryB -> fileContentsA This meant that the user kept all meta info for files, e.g. tags, window position, custom icons, etc. So when saving a new document you wro…

Common Lisp standard also strongly suggests this behavior for the :if-exist :supersede argument to #'open:

"The existing file is superseded; that is, a new file with the same name as the old one is created. If possible, the implementation should not destroy the old file until the new stream is closed."

http://clhs.lisp.se/Body/f_open.htm

I imagine they lifted this idea from some prior Lisps, which suggests it's a pretty old concept.

Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)

#94

Post by TeMPOraL »

skissane wrote: post #73
Animats wrote: post #59

I'd argue that UNIX-type file systems should offer several types of files: * Unit files. When you create a file and write it, it's not visible for other opens until you close it. If you open a file with O_CREAT|O_WRONLY|O_TRUNC, you create a new file, which replaces the old one on close. In the event of a program or system crash, or exiting via "abort" without closing first, the old file remains. So there's always on…

> * Temporary files. You can do all the file operations, and the file disappears on a reboot. It would be nice to be able to have a process tree own a temporary file, such that when the last process in the tree exits (not necessarily the process which created the file), the file is automatically deleted, rather than having to wait for the next reboot.

I would prefer it to not be the default; it's sometimes useful to keep temporary files in the event of a process crash, especially on a server that would then immediately restart said process. I had such a case very recently, and was thankful for the existing behavior of tmp files.

Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)

#95

Post by TeMPOraL »

sojmq wrote: post #88
Gene_Parmesan wrote: post #71

Earlier quoted context omitted.

The more time I spend as a dev, the more I realize that writing clear, simple, straight forward code is actually the greater challenge. When I started progressing beyond learning the basics, the sort of projects I was building were quite simple, so writing the simple code for them came to feel boring. I would read complex codebases and see all the fascinating tricks they employed and wished I was writing code like th…

Problem is when you have a shitty language like Python that does almost no optimisations so you're forced to write "clever" code if you want it to run reasonably fast.

If you must use Python, then pawn off the important work to C. Half of the reason this language exists for is easy FFI, and that's all the popular libraries get enough performance to be usable for non-toy applications.

Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)

#96

Post by IshKebab »

vbezhenar wrote: post #84
skissane wrote: post #73

Earlier quoted context omitted.

> * Temporary files. You can do all the file operations, and the file disappears on a reboot. It would be nice to be able to have a process tree own a temporary file, such that when the last process in the tree exits (not necessarily the process which created the file), the file is automatically deleted, rather than having to wait for the next reboot.

Can't you create file, unlink it and then fork as much as necessary? I think that OS will maintain reference counter for that inode and will delete it when all processes will close that handle (explicitly or implicitly with exit).

Sadly not because then the file doesn't have a filename, and there are a lot of things that only work with file names, not descriptors. Even basic stuff like std::fstream needs a filename.

Yes there are workarounds (library extensions, signals, /proc/fd, etc.) But you'd think a basic function like "delete this file when this process exits" wouldn't be too hard.

Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)

#97

Post by saagarjha »

jiggawatts wrote: post #43
bvinc wrote: post #29

I think he's got a point. As a developer I find myself in a different scenario. I'm usually trying to find out what exactly the 100% guaranteed way to do something is. Instead, I find incomplete documentation and different people with different opinions on what the guarantees are, and most people writing bad code that they assume will usually work. Just modifying a file in an atomic way requires a complicated dance o…

I do, but it's not a popular opinion. POSIX, and by extension, the classic 1960s-1980s era UNIX way of doing things just needs die a long overdue death. This stuff was designed at a time when every CPU instruction mattered, everything was optimised to death for frugality, and commands were abbreviated from "copy" to "cp" because ermahgerd two bytes is a huge saving! That mentality got us Y2K. This is an era where lat…

> commands were abbreviated from "copy" to "cp" because ermahgerd two bytes is a huge saving

Considering the number of times I type that command, I wouldn't be as quick to throw out the savings here.

Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)

#98

Post by mehrdadn »

Animats wrote: post #59

I'd argue that UNIX-type file systems should offer several types of files: * Unit files. When you create a file and write it, it's not visible for other opens until you close it. If you open a file with O_CREAT|O_WRONLY|O_TRUNC, you create a new file, which replaces the old one on close. In the event of a program or system crash, or exiting via "abort" without closing first, the old file remains. So there's always on…

Fun fact for those who don't know: Windows already has a database API. I've never used it, but it's called the Extensible Storage Engine (JetCreateDatabase, etc.).

Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)

#99

Post by michaelmrose »

Shish2k wrote: post #87
jiggawatts wrote: post #43

Earlier quoted context omitted.

I do, but it's not a popular opinion. POSIX, and by extension, the classic 1960s-1980s era UNIX way of doing things just needs die a long overdue death. This stuff was designed at a time when every CPU instruction mattered, everything was optimised to death for frugality, and commands were abbreviated from "copy" to "cp" because ermahgerd two bytes is a huge saving! That mentality got us Y2K. This is an era where lat…

I give it 2-3 more years of people saying "this is impossible" before Lennart does it and makes everyone mad

Nobody said it was impossible

Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)

#100

Post by fanf2 »

mehrdadn wrote: post #98
Animats wrote: post #59

I'd argue that UNIX-type file systems should offer several types of files: * Unit files. When you create a file and write it, it's not visible for other opens until you close it. If you open a file with O_CREAT|O_WRONLY|O_TRUNC, you create a new file, which replaces the old one on close. In the event of a program or system crash, or exiting via "abort" without closing first, the old file remains. So there's always on…

Fun fact for those who don't know: Windows already has a database API. I've never used it, but it's called the Extensible Storage Engine (JetCreateDatabase, etc.).

Is this the same storage engine that used to provide the back end for Visual Source Safe and Exchange? Which has a 2GB limit per storage unit and corrupts itself irreparably if you hit the limit?
Post reply on HN