Why does it take so long to build software? (2020)

"splitting it up" is only accidental complexity if one splits things up in services. If one has a monolith I would argue that "splitting it up", in that case known as 'improving coherence' and 'reducing dependency' is much easier and falls under essential complexity.

Generally, I would say that tools are a very big part of it. One example. Why did we go from svn to git again? O yes, git can do so much more. But now you actually have to do all of these 'so much more' things. This gives you a very beautiful commit history if you do it well but is a very beautiful commit history really that important? You tell me.... maybe it is and maybe it isn't and maybe it depends on the nature of the project....

I thought the original article was very poor. There's no real data, no references (except Fred Brooks), and the graphs are just nice shaded opinions based on nothing much.

It fails to ask why the complexity of tools and systems grows over time, and whether this is matched by improvements in UX.

It also fails to note that one of the goals of abstraction is to hide inherent complexity. And in fact this happens time and time again in engineering and product design - an insight that starts off requiring near-genius levels of original mathematical creativity is packaged into commodified tools that make it easy to do a certain job.

Sometimes the tools are aimed at engineers, and sometimes they're built for the public. But because no one expects to literally reinvent the wheel, they all package genius in their different ways.

Except software. In software teams keep working to reinvent wheels completely every few years. Some are arguably more refined than existing wheels. But they're all wheels.

They're not a new kind of thing - like an engine. Or wings.

So I agree. Software has mediocre lumpy wheels They're mostly back-references to existing software, and not so much attempts to commodify and simplify standard problems - like security, reliability, scalability, and UX.

There are packages for all of the above, but they're rarely clean and elegant - in the sense that someone has really understood the problem and designed a maximally effective but minimally complex automated solution.

Worse, there are no processes for efficiently abstracting a domain. And that's what's missing.

I think there is a cost disease in software where baseline requirements only increase depending on the maturity of a company. Accessibility, localization, scalability, permissioning, observability and alerting, reliable deploy and testing pipeline... all these things are table stakes for a new project at the company I work at, represent A LOT of complexity, yet an application delivered without them might have a nearly identical user experience...

That’s not even to mention the added burdens on the planning side of things, e.g. getting infosec and legal and compliance to sign off before breaking ground, gathering requirements from 10 different teams across the world, appeasing more stakeholders, etc.

The difference just in table stakes between a young startup and a mature company can easily account for an order of magnitude difference in effort required to deliver a feature, and it shouldn’t be surprising.

Edit: and this is why I weep when I see mature companies under-investing in developer tooling.

Writing software is basically going in debt, so you should write the 20% of software that will deliver 80% of the value. The problem begins when not considering costs and then being "surprised".

What is the value of the current pace of change though? We are using billions of developer hours every year to keep up with all the changes, where is it going, why do we need to spend so much time changing things? Changing things doesn't just force people to rewrite, even worse it forces people to relearn instead of achieving mastery.

How many percent of developers understand how everything in their stack works to such a degree that they'd be surprised if there are any problems when they test things? Basically nobody, yet it isn't really an unachievable goal. Why can't a web developer reach that point after 5 years of full time? Simple, it is all the damn changes to everything that happens constantly. It is so bad that most seems to think it is impossible to master anything in software engineering, yet if you just pin down the versions of all your dependencies and work like that for a few years everything will be crystal clear, that is how the human brain works.