That title would explain most of the industry.
Slaaaaaaaammmmmmmmm.
Filesystem devs should aim to make “badly written” app code “just work” (2009)
31–40 of 117 posts
Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)
#32"Anybody who wants more complex and subtle filesystem interfaces is just crazy. Not only will they never get used, they'll definitely not be stable." I think there is a more universal truism here - that "complex and subtle" are sources of pain, problems and headaches. I want to write "cool" and "magical" code as much as the next person, but that's the stuff that I look at later WTF because I am no longer in the same…
I like your description because it (accidentally?) likens the state of mind of wanting to write "cool" and "magical" code with being high on drugs. In my career I've made that connection many times. This team is not interested in some engineer having an "awesome trip" through arcane abstractions, we're interested in shipping stuff that works.
Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)
#33Told you: Perl is built by the gods.
Perl has subtle complexity just like everything else. But it also has taint mode, which depending on the way you squint is either a great example of "security that just works", or a great example of "you must use X method to get good security".
I vote for the second option :-) IMO taint mode is a great example of what Linus is talking about. Too subtle.
I cut my teeth on Perl and actually believed the books when they said you should always use taint mode when touching data that came e.g. over the network. I wrote all my web code under -T (-wT actually but you get the idea).
Then one day I went to drop in some full text search via a then popular library (Plucene). And what do you know, Plucene would not work under taint mode, because it was not developed under taint mode. The maintainer would not accept my simple patch essentially because he did not understand that you could not untaint without a regex somewhere (i.e. he did not know how taint mode worked at a basic level). So I maintained a patched version of that lib privately. Only to later hit the same issue with another popular library.
So I had to stop using taint mode. If it’s just an option — even one aggressively marketed in O’Reilly books back when people actually mostly read O’Reilly books to learn various systems — it’s not going to win much adoption.
Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)
#34"Anybody who wants more complex and subtle filesystem interfaces is just crazy. Not only will they never get used, they'll definitely not be stable." I think there is a more universal truism here - that "complex and subtle" are sources of pain, problems and headaches. I want to write "cool" and "magical" code as much as the next person, but that's the stuff that I look at later WTF because I am no longer in the same…
And that’s why reading Golang is much more straightforward than Rust which is way more verbose/flexible.
Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)
#35Quite a bizarre read considering Linux's decision to e.g. overcommit memory makes 100% correctly written code break nondeterministically...
It seems to me that a basic requirement for not being an idiot is that you recognize hard problems as being hard, and I see a lot of supposedly smart people declaring hard problems are easy because they're just ignoring tradeoffs or aspects of an approach that undermine it. I'd think "everybody" knows, certainly I would expect Linus to know, about Postel's law and the subtle ways it ends up causing problems. Whenever…
What do you think the "insoluble" problem is here?
Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)
#36The thread title omits a crucial word: "Filesystem" (as in "filesystem people", not just "people"). The point he is making is that filesystems are supposed to be utterly reliable; applications should not have to take extreme precautions to avoid having the filesystem lose their data. And the fact that practically nobody actually takes any precautions, let alone extreme ones, is strong evidence that programmers do in…
More to the point he's saying crappy code that uses the filesystem should just work if possible. Don't make it worse than it is. And don't expect anyone's going to fix their code to use your new and improved API. Because they won't.
Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)
#37Is there a point though, when badly written code becomes so hard to maintain and improve, that people would just avoid doing that altogether? The reason "badly written" works for Open Source is that if code is useful - there will be someone in the future who will refactor it. In proprietary setting that only happens when the fate of the company itself (or a large chunk of the business) is at stake. Otherwise stagnati…
It's easy to pretend that "badly written" code is intentional or desirable, because then when we write it, we can excuse ourselves by saying, "BUT IT WORKS!"
That's why this idea is popular, and your comment is controversial. You're taking away a crutch that many of our peers cling desperately to as a way to justify their shortcuts and poor decision making.
Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)
#38Earlier quoted context omitted.
Yeah. So many api writers aim to force clients do all the heavy lifting. The whole point of a good api is that it reduces heavy lifting. Anyone can write pass through apis that don’t do anything.
> The whole point of a good api is that it reduces heavy lifting. Isn’t that the opposite of what Torvalds is saying? He seems to be arguing for simplicity. APIs that do a bunch of magic for you are the opposite of simple and tend to be mountains of subtle bugs and unexpected behavior.
I think that lies in the art of developing the API in the first place. It should give you enough primitives that it gets the job done balancing the responsibilities it advertises with the cognitive load on the developer to use it correctly.
Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)
#39Earlier quoted context omitted.
The difference is that people do tend to check return values of write() (in cases where it actually matters), but not check the return value of malloc(). Since Linux must work with the code which exists, and since most programs tend to malloc() memory but never use it, the overcommitting practice begins to make sense. A twisted and ugly sense, but that is the world we live in.
That's at best a reason to allow over-commitment to be enabled or disabled on a per-program basis, not a reason to force it on the whole system so that it becomes outright impossible to write correct code.
I think this is why it's not per-program:
Say you have 16GB of RAM. You disable overcommit for your database, and you run it, and it allocates 14GB. But Chrome has overcommit enabled (why are you running Chrome and a database on the same box? Dunno, never got an answer from that engineer...), and it happily allocates 23GB. Later, Chrome uses up all the available memory, and the database is sad. But it's not Chrome's fault, it didn't know it wasn't allowed to use it all up! So then you redesign overcommit so there's a "no-overcommit pool" with priority, and an "overcommit pool". You end up with only 2GB of overcommit pool for the kernel + userland + Chrome, so Chrome just gets killed early and often. Might as well have just disabled overcommit entirely, if random apps are going to die anyway.
(Incidentally, cgroups allow setting soft and hard memory limits, so you can impose particular memory limits on arbitrary applications to have more determinism)
Re: Filesystem devs should aim to make “badly written” app code “just work” (2009)
#40The thread title omits a crucial word: "Filesystem" (as in "filesystem people", not just "people"). The point he is making is that filesystems are supposed to be utterly reliable; applications should not have to take extreme precautions to avoid having the filesystem lose their data. And the fact that practically nobody actually takes any precautions, let alone extreme ones, is strong evidence that programmers do in…
The title should be changed, he's saying literally the opposite of what the title implies.