Live data from Hacker News

Filesystem devs should aim to make “badly written” app code “just work” (2009)

lwn.net

71–80 of 117 posts

Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)

#71
post #5

"Anybody who wants more complex and subtle filesystem interfaces is just crazy. Not only will they never get used, they'll definitely not be stable." I think there is a more universal truism here - that "complex and subtle" are sources of pain, problems and headaches. I want to write "cool" and "magical" code as much as the next person, but that's the stuff that I look at later WTF because I am no longer in the same…

The more time I spend as a dev, the more I realize that writing clear, simple, straight forward code is actually the greater challenge. When I started progressing beyond learning the basics, the sort of projects I was building were quite simple, so writing the simple code for them came to feel boring. I would read complex codebases and see all the fascinating tricks they employed and wished I was writing code like that. It felt like, "Those are the smart programmers. I should emulate them."

As the complexity of the projects I work on has increased (especially in the time since I became a part of a professional dev team), I've come to realize that most of that tricky, complicated-looking code I had read years earlier was actually kind of the easy way out. When you are just trying to get stuff done, the complexity of your code mirrors the complexity of the project. There's no effort put in (and often no time to do so) to create a smoother interface to shelter the code from the complexity of the task at hand.

It's much more interesting to me now to be faced with a complex task and to figure out how to make the code simple and clear. Elegant, well-designed libraries/APIs belie the challenge in writing them. The code looks so simple that it feels immediately obvious as you're reading it. I've come to realize that reading a code base that seems dead simple -- so simple anyone could instantly understand it -- is actually often inversely related to the difficulty in creating it.

Yes, necessary complexity does exist, and sometimes there's no way around doing something fairly nasty in your code. But as an ideal to strive for, I find 'simple' to be fascinating.

Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)

#72

Earlier quoted context omitted.

> code in comments that isn't being maintained drift even further. This is also true of "its in the repo" and that doesn't get hit by "find and replace". I don't think it should be done for EVERY case, rather the cases where clarity is replaced with something that is ambiguous but meets another need. Performance hacks are notorious for being "ugly" and "magical" and having a readable companion piece would make sense.…

To add to my earlier response: if I needed to achieve the performance critical cleverness required solution and felt it needed a human readable code sidekick, I would include both as real running code alongside each other, both of which running the same set of tests, and both of which required to be run by whatever CI/automated tests are used by the project. Comments are for humans to read, not machines.

I like this approach, it would cover 80-90% of the cases where it would provide value!

Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)

#73
post #59

I'd argue that UNIX-type file systems should offer several types of files: * Unit files. When you create a file and write it, it's not visible for other opens until you close it. If you open a file with O_CREAT|O_WRONLY|O_TRUNC, you create a new file, which replaces the old one on close. In the event of a program or system crash, or exiting via "abort" without closing first, the old file remains. So there's always on…

> * Temporary files. You can do all the file operations, and the file disappears on a reboot.

It would be nice to be able to have a process tree own a temporary file, such that when the last process in the tree exits (not necessarily the process which created the file), the file is automatically deleted, rather than having to wait for the next reboot.

Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)

#75
post #5

"Anybody who wants more complex and subtle filesystem interfaces is just crazy. Not only will they never get used, they'll definitely not be stable." I think there is a more universal truism here - that "complex and subtle" are sources of pain, problems and headaches. I want to write "cool" and "magical" code as much as the next person, but that's the stuff that I look at later WTF because I am no longer in the same…

I am currently trying to wring maximum performance from Cloud TPUs. This comment really resonated with me, because Cloud TPUs are complex, subtle, overly complicated, and have twenty opaque ways that you can ruin your performance. Compare that with a GPU. It took decades, but (at least for nvidia GPUs) they have finally reached the point where you can be at least reasonably certain that high-performance is the defaul…

Anecdotally, I have seen 50-100x (seriously) increases when moving from a V100 to TPUv3. To be fair, some of that is batch size increase. This was using the models in https://github.com/tensorflow/tpu/tree/master/models/officia.... On the other hand, a lot of those models are broken in some way and need fixing before running, caveat emptor.

Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)

#77
post #59

I'd argue that UNIX-type file systems should offer several types of files: * Unit files. When you create a file and write it, it's not visible for other opens until you close it. If you open a file with O_CREAT|O_WRONLY|O_TRUNC, you create a new file, which replaces the old one on close. In the event of a program or system crash, or exiting via "abort" without closing first, the old file remains. So there's always on…

> * Temporary files. You can do all the file operations, and the file disappears on a reboot. It would be nice to be able to have a process tree own a temporary file, such that when the last process in the tree exits (not necessarily the process which created the file), the file is automatically deleted, rather than having to wait for the next reboot.

This is mostly about defining behavior for the bad cases - program abort and system restart. Those are classically undefined behavior, which causes problems.

Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)

#78
post #5

"Anybody who wants more complex and subtle filesystem interfaces is just crazy. Not only will they never get used, they'll definitely not be stable." I think there is a more universal truism here - that "complex and subtle" are sources of pain, problems and headaches. I want to write "cool" and "magical" code as much as the next person, but that's the stuff that I look at later WTF because I am no longer in the same…

I don't think you can judge code just by how readable it is, and you can't frown on what seems cool and magical all the time. Sometimes "clean and readable" means that complexity is pushed outwards onto dependencies or you've simply created an inelegant solution that doesn't consider the whole problem domain.

A perfect C++ program that outperforms a simple looking python program which does the same thing isn't worse because it's harder to read or to write, or better because it's faster, it's just different.

If you need to optimize for readability, so be it. If you need to optimize for performance, so be it.

If you write code executed trillions of times in a trillion places, where power use and the expense of hardware is dependent on the performance of the software, I think it's okay for it to be as complex as it needs to be to be very optimal, at the expense of readability and other things. It should be well annotated, sure. That's always a reasonable thing to expect except in rare, extreme cases where you don't even have the time to type out the comments.

It's important to weigh the relative importance of simplicity vs speed vs memory frugality vs portability vs maintenance difficulty (dependencies and tool choice factor in here) and so on to determine the best approach.

Dogma about how best to code in a general sense is only helpful if you don't want to think or talk about what's the best fit.

Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)

#79
post #51

Earlier quoted context omitted.

You really like leaving in commented code? I used to prefer that too but then at work we had a policy of taking out unused code. I actually think it's cleaner so I've been doing it in my own code and rarely regret deleting something. But when code (my own or someone else's) is more chaotic I do find it to be useful to leave bits and pieces lying around.

I think this is about the rare cases where you have multiple competing implementations that become better or worse depending on small changes elsewhere or when a simple initial implementation can serve as documentation for a carefully optimized subsequent implementation. I tend to think there is always a better way, say creating a module that contains both implementations as alternatives, but if this is the worst war…

I would go for a comment like "a simpler naive implementation is in commit A0348C. It ran approx. 2.5x slower in 2019."

Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)

#80
post #59

I'd argue that UNIX-type file systems should offer several types of files: * Unit files. When you create a file and write it, it's not visible for other opens until you close it. If you open a file with O_CREAT|O_WRONLY|O_TRUNC, you create a new file, which replaces the old one on close. In the event of a program or system crash, or exiting via "abort" without closing first, the old file remains. So there's always on…

Nagle, I believe I last read your comment on this (https://news.ycombinator.com/item?id=13964053), progress has been made .

We have the temporary files you're asking for. https://lwn.net/Articles/619146/

With O_TMPFILE, you can also write new data, and then automatically replace a file on disk.

Post reply on HN