"Software Engineering" Is Not Engineering (2005)

I could be stretching, but to me, this is very conscious and intentional work, perfected through experience and there are many way to get it wrong. What exactly would we call this?

There are programmers who do that, and I think it's the way to make good software, to care about what the machine actually does that is, and not just if it's easier to write one line instead of five (which is where we got this bloated framework mess from)... but it's hardly some sort of standard for the profession, much less the minimum.

With engineering (I would assume), you rarely see, say, some sort of tool (made by an engineer, that is) that has half of another tool sticking out of it in a spot "where it doesn't matter" because they preferred to ship faster. You'll never see a tool that just weighs 100x more than it needs to because hey who cares, it's not like the person who made and sold that thing actually has to use it.

With software, costs are externalized left and right because to see them, you'd have to actively think about stuff, and customers don't know anyway so anything goes, unless you go too far. But if a washing machine is the size of half a city, everyone notices.

> With “classic” engineering at least you have the immutable laws of physics to judge your work, but with software we have no such luck - software is infinitely pliable in ways equivalent to bending the laws of physics in classical engineering. Your bridge may not be sound at one Earth gravity, or your software might not work reliably with a gigabyte of memory, but it’s like we can place your bridge under half G by giving the software twice as much memory. And we can do all that after building our “bridges”.

In that example, software isn't actually "plied" (if that is a word?) though, the amount of RAM is. So the software stays as is, mediocre. And art, like engineering, often benefits from constraints. Imagine if the solution to being a shitty artist would be to just drug the audience so they're happier and like whatever they're seeing more. Even if it benefited them, even if you could tell them and they'd be fine with it, that's not the way to hone a craft, that just seems obvious on the surface to me.

I don’t think it’s about science. It’s about the scientific method. From wikipedia

  The scientific method is an empirical method for acquiring knowledge through careful observation, rigorous skepticism, hypothesis testing, and experimental validation

That said, there's been a lot of changes since this was written that was relevant to the point being made, so I'm not sure I necessarily disagree with the article as written at the time. If we take the metric I used in that blog post and asked "what if we were handed current-gen AI in 2006, how well would it do?", my answer is that it would do noticeably less well. This is even more true if we instead go back to 2006 and use current hardware tech but have to train on what was available at the time.

Source control was fairly popular at that point, although it hadn't penetrated everywhere yet. I don't recall a lot of curmudgeons complaining that Real Programmers Don't Need Source Control at the time, it was pretty obvious that a record of what was happening in the source code base was useful. However, IIRC, in the open source world we'd mostly have been on cvs. svn was well into the process of eating it but still had a ways to go yet. "git" had just barely been born and it would be years before it was a serious force.

Beyond that, a lot of stuff I take for granted in the engineering domain was either immature or non-existent. I was on the cutting edge bothering everyone about the importance of unit testing. There were communities like Perl that had a strong culture of it and a lot of support, but still a lot of people who would use it without that and a lot of language communities without the culture. A lot of projects banged together their own haphazard solutions, but a lot of projects just went without. No devops. Metrics was something I'd never really heard of. Logs probably went to your disk and the idea of unifying them was just at the beginning. QA was much less established. QA was generally not a team that would be using much automation, it would be all hand testing. CI/CD wasn't a term yet because hardly anyone had anything even resembling it. Bug trackers were still pretty bad and not everyone used them.

I'm sure a lot of people will go "yeah, but I was programming in 2006 and I had all those things", because there were certainly teams that did. Go look at Microsoft Window's dev team, for instance, and they'd have most of what we'd consider a modern development stack, albeit with some quirks we'd find odd. It's not that none of this existed, it's that it wasn't considered just the baseline for a project to be minimally competent the way it is now. And rather than the programmers having an abundance of free options and an even larger abundance of paid and hosted options, to the point it's hard to poke through them all, they're using one of a few very expensive vendors or they put it together themselves.

Introducing an AI into a code base of tens or hundreds of thousands of lines, with minimal testing, trained on the code practices of that era... a modern programmer could probably still work with it. You could tell it about unit testing and it would know what you meant, it would probably just need to be prompted to do it. You could put together unified logging with some work. You could solve the problems... but your solutions would basically be putting in those 2026 guard rails. You'd have the same uphill battle convincing people of the time that this wasn't just a nice-to-have but a bare necessity, though it would be a much less uphill battle when you have a tool that can do it with a lot less effort, making the cost/benefit tradeoff a lot more appealing when the cost is so much lower. But a lot of programmers of the time would make even larger messes even more quickly with AIs happy to do so if you didn't give them some guidance on how to use it. They would, of course, figure it out eventually, since "they" are also "us", and we collectively figured this out without AIs helping us along. But boy oh boy would there be some messes made first.

I think some element of the problem is that, yes, programming was once a wild and wooly frontier of cowboys and crazy people doing crazy things. But one step at a time, the field has grown up, and a lot of people haven't noticed and still have this "cowboy programmer" idea in their mind. I don't know exactly when you might want to say the field became a real engineering field, with its own practices based on the local cost/benefit tradeoffs and its own procedures and its own "bare minimum for competence" standards. That partially depends on your own definitions. Some people who want to put the "bare minimum" at something like "uses dependent types and carries proofs for every line of code" might say we're not there yet, but you can always set the bar higher. But I think a lot of people have not looked around and realized that, yeah, actually, we have crossed that bar.

Personally I put it around a decade ago. But I think we're there. We can stop wringing our hands about not being "real engineers" and stop looking over at what the "real engineers" do. We have better solutions for our problems then any amount of copying what they do could give us, just like they can't just lift our practices and apply it to their work because it doesn't fit their world. It's gotten to the point where dropping what we've got in hand now and trying to jam some other "real engineering" discipline's tools in would be a rather substantial regression. We can chill out a bit.

We can also be glad that all the calls to try to systematize the field so it would be "real engineering" didn't go and lock in the 2006 idea of what "good software engineering" is so that we got stuck to it and we couldn't progress to where we are today. Two years ago I would have said maybe such an effort could do something useful, but then AI came along and overturned the apple cart and now, once again, I would be very suspicious of any attempt to lock in July 2026 practices as The One Correct Way To Write Software For The Next 100 Years.