The collapse of complex software

Fix the tools, and you start to fix the code. Unnecessary terminology, confusing APIs, superfluous patterns...it's like watching someone blindfolded in a dark room trying to justify the high price tag on their computer science degree.

After years of running my mouth, I proved my rants about the unnecessary complexity by building my own JS framework [1] and I'll never look back. The emperor is bare-ass naked.

[1] https://github.com/cheatcode/joystick

Among other things, it suggests that there are some incentives for complexity: That engineers just enjoy complexity, and that making simple designs might not be good for the respect of your peers/bosses/career: "A simple design might make it seem like you’re not really doing your job."

While that's true, I think the essay isn't clear enough about the fact that simplicity is just plain hard. Complexity is sort of the "natural" outcome of "adding things". It takes time, effort, and skill to create software that can do what is asked for it while maintaining an internal simplicity. Even if everyone involved really wants it. I do not think software complexity comes only, or mainlly, from people not wanting it enough (explicitly or implicity), from incentives to complexity. It's just plain hard. It's a mistake to think just wanting it harder (or having the right incentives) is sufficient.

It may be the key challenge in software engineering -- how do we make software that does everything that's asked for it (powerful, complex things), while maintaining an internal simplicity? You can not accomplish it by cargo-culting any "design patterns" or other approaches. It's still a skilled craft. That comes from years of experience with the intent of developing this skill (years of experience alone aren't enough), looking at other people's code who are also trying to do it, and just some aptitude. And domain knowledge, if you understand the nature of the problems you are working on better, you can design simple solutions for them better--contrary to some current practices treating developers as interchangeable.

Yes, you need your organization to allow you to do it (including taking the more time it will take), but that's necessary but not sufficient for the outcome.

Just an example: around here, most people have a first (given) and a last (family) name. If I don't model that as separate, I have trouble interfacing with other software. If I do, I have trouble with people from other cultures that don't follow that convention. Storing both risks the data going out of sync. What's the "right" way to store person names? There doesn't seem to be a simple solution.

Another example: we model physical cables (both for power grid and for data transmission) in our CMDB. All works fine, until you suddenly have a Y-shaped cable with three connectors that doesn't fit into your data model. The real world always has these 1% of cases that don't fit the general pattern; if you focus on the 99%, the 1% make trouble. If you focus on modeling every case, you have 10x the complexity, even for the simple case.

And then there are things that are moderately complex and security critical, like password recovery workflows. We haven't really found a way to reuse these among different technologies. Like, if you once figured out the perfect password reset workflow with Ruby on Rails, and your next job uses Python + Django, you're back to square one.

If somebody has a good idea for how to tackle these problems, please let me know!

But not because some esoteric systemic property of complex systems, but just because working projects get deprived of maintainers and eventually there's nobody that understands WTF original programmers intended and how to fix the holes.

So incomprehensible portions of the system are circumvented or replaced with something that just works. Actually the same happens in societies, unless some group sabotages the workaround.