Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Filesystem devs should aim to make “badly written” app code “just work” (2009)

lwn.net

Post reply on HN
Print view View original

51–60 of 117 posts

Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)

#51

Post by jrumbut »

pizzaparty2 wrote: post #42
zer00eyz wrote: post #5

"Anybody who wants more complex and subtle filesystem interfaces is just crazy. Not only will they never get used, they'll definitely not be stable." I think there is a more universal truism here - that "complex and subtle" are sources of pain, problems and headaches. I want to write "cool" and "magical" code as much as the next person, but that's the stuff that I look at later WTF because I am no longer in the same…

You really like leaving in commented code? I used to prefer that too but then at work we had a policy of taking out unused code. I actually think it's cleaner so I've been doing it in my own code and rarely regret deleting something. But when code (my own or someone else's) is more chaotic I do find it to be useful to leave bits and pieces lying around.

I think this is about the rare cases where you have multiple competing implementations that become better or worse depending on small changes elsewhere or when a simple initial implementation can serve as documentation for a carefully optimized subsequent implementation.

I tend to think there is always a better way, say creating a module that contains both implementations as alternatives, but if this is the worst wart on the codebase it is probably a good codebase.

Just leaving giant blocks of commented out, trivial to recreate code as the system evolves is annoying. But I do think that the rule of not leaving old code in as comments is one where the bar to breaking it shouldn't be too terribly high. Personally I've gone overboard following it in the past, and all it did was hurt productivity. A few comments blocks aren't the end of the world.

Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)

#52

Post by gugagore »

zer00eyz wrote: post #5

"Anybody who wants more complex and subtle filesystem interfaces is just crazy. Not only will they never get used, they'll definitely not be stable." I think there is a more universal truism here - that "complex and subtle" are sources of pain, problems and headaches. I want to write "cool" and "magical" code as much as the next person, but that's the stuff that I look at later WTF because I am no longer in the same…

The clean and readable version can be kept, but not in comments. Why not keep it as a reference implementation, and test the new thing against it?

Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)

#53

Post by ijiiijji1 »

A robust interface does what the user intended, not just the bare minimum to fulfill a specification. And it introduces the fewest novel quirks, rough edges and gotchas possible. Because that's what people want: things to be handled lower down with least surprises, including "just works" gracefully as possible in the face of real-world errors. This usually means handling some corner-cases lower down with special treatment so as to not burden the caller.

Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)

#54

Post by naniwaduni »

capdeck wrote: post #10

Is there a point though, when badly written code becomes so hard to maintain and improve, that people would just avoid doing that altogether? The reason "badly written" works for Open Source is that if code is useful - there will be someone in the future who will refactor it. In proprietary setting that only happens when the fate of the company itself (or a large chunk of the business) is at stake. Otherwise stagnati…

> there will be someone in the future who will refactor it

maybe abstractly

in practice, this never happens because the code will cease to be considered useful first

Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)

#55

Post by zielmicha »

dooglius wrote: post #44
peterwwillis wrote: post #25

Earlier quoted context omitted.

I think the difference between "don't implement workarounds" and "use overcommit" is that the kernel can try to be clever, but userland should not have to be clever. The kernel is supposed to just make things work for userland. I think that's why overcommit exists. It's generally not easy to re-design all userland applications to deal with difficult memory management problems in a complex system, but it is easy to ju…

The problem is that userland has to be really clever because of overcommit... that's what the whole thread is about! The reason people want complex things like fsync and barriers is that the OOM killer has normalized the bad idea that applications should behave well when suddenly SIGKILL'd, and this is way harder than looking at the return value from malloc. When a program I write has elevated permissions, which fort…

You don't need fsync if you only care about your user process being killed. It protects only against power loss/kernel crash - the filesystem is not running as user process, so it can't be killed.

Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)

#56

Post by zer00eyz »

eyelidlessness wrote: post #50
zer00eyz wrote: post #5

"Anybody who wants more complex and subtle filesystem interfaces is just crazy. Not only will they never get used, they'll definitely not be stable." I think there is a more universal truism here - that "complex and subtle" are sources of pain, problems and headaches. I want to write "cool" and "magical" code as much as the next person, but that's the stuff that I look at later WTF because I am no longer in the same…

I agree completely don't do magic, keep it clean and simple. I disagree with leave the thing you replaced commented. It's not just that you can find it "in the repo", it's that things drift over time. Prose comments have that reputation but code in comments that isn't being maintained drift even further. No one even has any intention of maintaining commented code except if it happens to hit find and replace, and even…

> code in comments that isn't being maintained drift even further.

This is also true of "its in the repo" and that doesn't get hit by "find and replace".

I don't think it should be done for EVERY case, rather the cases where clarity is replaced with something that is ambiguous but meets another need. Performance hacks are notorious for being "ugly" and "magical" and having a readable companion piece would make sense.

To clarify I don't think it replaces "prose" but should exist along side it, and be mixed into it.

Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)

#57

Post by eyelidlessness »

jiggawatts wrote: post #43
bvinc wrote: post #29

I think he's got a point. As a developer I find myself in a different scenario. I'm usually trying to find out what exactly the 100% guaranteed way to do something is. Instead, I find incomplete documentation and different people with different opinions on what the guarantees are, and most people writing bad code that they assume will usually work. Just modifying a file in an atomic way requires a complicated dance o…

I do, but it's not a popular opinion. POSIX, and by extension, the classic 1960s-1980s era UNIX way of doing things just needs die a long overdue death. This stuff was designed at a time when every CPU instruction mattered, everything was optimised to death for frugality, and commands were abbreviated from "copy" to "cp" because ermahgerd two bytes is a huge saving! That mentality got us Y2K. This is an era where lat…

Speaking as someone whose job is primarily building high level APIs, this is unachievable. Those high level APIs satisfy the work the vast majority of people want or need to achieve, but some tasks require greater control over the details those APIs achieve, and all tasks require those details to otherwise be right.

POSIX might not be the right lower level API, and maybe it can be replaced with something that achieves those details better. But "done bang just do it" is simply not how computers work. Software is how you achieve that, and generally by building simpler abstractions on top of more brittle ones.

"How hard can this be?" is the right question. It's very damn hard.

Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)

#58

Post by eyelidlessness »

zer00eyz wrote: post #56
eyelidlessness wrote: post #50

Earlier quoted context omitted.

I agree completely don't do magic, keep it clean and simple. I disagree with leave the thing you replaced commented. It's not just that you can find it "in the repo", it's that things drift over time. Prose comments have that reputation but code in comments that isn't being maintained drift even further. No one even has any intention of maintaining commented code except if it happens to hit find and replace, and even…

> code in comments that isn't being maintained drift even further. This is also true of "its in the repo" and that doesn't get hit by "find and replace". I don't think it should be done for EVERY case, rather the cases where clarity is replaced with something that is ambiguous but meets another need. Performance hacks are notorious for being "ugly" and "magical" and having a readable companion piece would make sense.…

> This is also true of "its in the repo" and that doesn't get hit by "find and replace".

It's very much not! The thing "in the repo" is of exactly the same vintage as the thing in the next commit. The thing "in the comments" is preserved in amber next to a growing thing that no longer matches it. Inevitably to understand the thing "in the comments" you have to go back to commits of the same age.

Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)

#59

Post by Animats »

I'd argue that UNIX-type file systems should offer several types of files:

* Unit files. When you create a file and write it, it's not visible for other opens until you close it. If you open a file with O_CREAT|O_WRONLY|O_TRUNC, you create a new file, which replaces the old one on close. In the event of a program or system crash, or exiting via "abort" without closing first, the old file remains. So there's always one completely written file. Creating a unit file is an atomic operation. Most files are unit files. (This isn't original with me; it comes from a distributed UNIX variant developed at UCLA in the 1980s.)

Replacing an existing file currently requires elaborate renaming gyrations which vary from OS to OS and file system to file system. At least for Linux, this should Just Work in the normal case.

* Log files. If you open a file with O_APPEND, you can only write at the end. The file system should guarantee that, even after a crash, you get the file as written out to some previous write. If you call "fsync", the recovery guarantee should include everything written up to the "fsync" point. No seeking backwards and overwriting on a log file.

* Temporary files. You can do all the file operations, and the file disappears on a reboot.

* Managed files. These are for databases and such. They have some extra API functions, for async reads and writes and commits. Async I/O should have two callbacks - "the data has been taken and you can now reuse the buffer", and "this write is definitely committed and will survive a system crash" That's what databases really need, and try to fake with "fsync". Only a few programs will use this, but those are important programs.

Post reply on HN