The Essence of Programming (2021)

Author of the article here:

"Programming is not a craft." It is a craft and ought to be treated as such. Not doing so leads to the mess in software we have today. Software that is no more complicated than software from 20 years ago but runs 100x-1000x slower than its older counter part. This is the direct result of not understanding what art and craft of programming is and not treating it as such.

I highly recommend reading the previous article to this one posted (Pragmatism in Programming Proverbs) to get a better understanding of what I am expressing, since "The Essence of Programming" is a sequel to that article.

The difference as I see it: computer science (sorry, "informatics") is a mathematical discipline, and hence tends to concern itself with, out of a given class, the minimal (and maximal, when existent) object(s). Programming is an engineering discipline, and hence tends to concern itself with, out of a given class, the intervals within that lattice that are optimal by some suitability function.

In principle, the suitability function would be evaluated over the entire lattice; in practice, that function, whether explicitly or implicitly, includes a strong weight for "distance from existing solutions". In either case, this split in focus between the interior and the boundaries of the solution space means that programmers are often highly concerned with specific details that do not even appear (because they have been abstracted away) in the objects with which informaticians work.

As an example: theory people love to use 1-ary trees (induction steps cost nothing in proofs, but cases are expensive) and they will use 2-ary trees (sometimes even without pressure to sympathize with the machine) but systems people and programmers use k-ary trees (where, if it's been determined by measurement and not by compatibility, k depends upon "the" bandwidth-delay product between the storage hierarchy levels for which the tree is optimized ... or at least what the bandwidth-delay product had been at the time of writing).

Author of the article here:

All I ever get from beginners are questions about the "best tool" by which they usually mean "a set of rules and procedures so that I don't have to think about the problem".

> The person is trying to ask the question in a way which suggests that he or she does know how to solve the problem in ways that are not so great, and is just looking for a better way.

It's pretty much always the exact opposite since they rarely understand the problem that they are trying to solve to begin with. And that's the issue! They want a "tool" to just do it for them, even if that tool is not even appropriate for the task at hand. And the questions asked are usually in the form of an XY problem: "how can I use X to do Y?". You might be able to bodge the tool X until problem Y is solved but that doesn't even mean it is appropriate to begin with.

For example: I get a lot of questions at the moment about Entity Component Systems (ECS) and asking what is the best way to make one (or which to use) because they want to make a game. But their games don't even require one in the first place since they have such a low number of entities and components that an ECS would probably be detrimental to them in the first place (performance and productively). But to clarify, if the individual wants to just learn about the topic rather than apply it, then I usually recommend them to learn about relational databases first and how they are implemented.

Many people want to use a tool because they've heard someone else (usually a semi-famous programmer or a big company) using it and then following it in a "flash fad" rather than asking whether or not it is even appropriate to use. I will state that this perfectly _rational_ to do by the way since you are trying to outsource wisdom to others that appear to have it. However, programming as a profession is a best 70 years old, and as a result there has been not been enough time for evolutionary selection pressure to reveal the wisdom, so many people (especially beginners0 are effectively walking around following whatever the latest Fad is hoping for wisdom and a set of rules to follow.

> People in this type of field sometimes have a hard time admitting they don't know how to do something. Particularly if they are not such beginners and have a track record of solving problems on their own, which has become part of their self-image.

Humility is a virtue but also a very difficult one possess. For beginners reading: don't be afraid to make mistakes and look foolish. It is a heck of a lot better to look ignorant but honest than to look ignorant but arrogant.

I highly recommend reading the previous article to this one posted (Pragmatism in Programming Proverbs) to get a better understanding of what I am expressing, since "The Essence of Programming" is a sequel to that article.

I just got my revelation on this when I got to writing scripts to manage servers.

Since I am developer I was always starting from position where I need "reusable and flexible" script. This approach got me stranded in doing stuff like updating TLS certificates by clicking in IIS settings - because I did not want even to start if I did not have "perfect script".

At some point I went "OK I am just going to do script for this single server" - well script is reusable because it is simple and for another server I just change variables inside of the script. It saves me a lot of time already.

Lots of business line software does not need flexibility but developers tend to have need to build a framework because somehow they feel like they are not real developers.

I actually agree with GP.

Counterpoint: “the mess in software that we have today” (we at least agree on this) is precisely generated by the fact that people treat programming as a “craft”, where everytime a (apparently) new problem arises, people feel the need to “craft” together some new framework or way of doing things, only looking at the small problem at hand and without considering the bigger picture.

In other words, people treat programming as an artisanal activity instead of what it should be: an engineering discipline and possibly still a scientific pursuit.

Let’s say this: Programming could be approached as a craft by the people who apply it to real world problems. BUT, there is a very different job, which is the one of the Computer Scientist (the name has a meaning), which should instead treat programming as a science. This is currently not happening, or not fast enough, exactly because so-called computer scientists treat programming as a craft, without truly exploring the fundamentals.

I have expressed this view several times in past comments here on HN, but we are still far from having a proper, theoretically grounded framework to do coding, and crafting more short-sighted solutions to pile up on top of each other won’t help…