Ask HN: Is Accidental Complexity Growing Faster Than Essential Complexity?

However, simultaneously, I'm seeing a great deal of technical complexity being introduced to systems: microservices and SPAs (with all the attendant web stack complexity) being the most common. A lot of it seems to be driven by a need for developers to keep their CVs shiny, or because smart developers want cool things to play with, or because of "Google envy". There seems to be very little engineering justification for these approaches in most cases, and technically simpler approaches would work better.

I've been thinking about Fred Brooke's Mythical Man Month, in which he described 2 forms of complexity: Accidental Complexity as being technical in nature, while Essential complexity relating to the problem domain, and being more intractable.

Although Essential complexity remains stubborn in business systems, I believe we are slowly getting to the point where it is being tamed through more responsive (agile) dev practices, leading to happier customers who get software that does what they expect. On the other hand, I feel that accidental complexity is exploding, and for the first time in my career, it is growing at a faster rate than essential complexity, and masking a lot of the gains we are getting from improved processes. Is this a fair observation, or am I just jaded?

In my mind you have two possibilities:

1. The most ideal is to "pick the right solution for the problem." Meaning, you will analyze the problem and do the right thing, e.g. content mainly? server side rendering. web application? SPA.

2. Do what you know, consistently. In many cases, this means using an SPA basically all of the time, whether or not it's necessary. This also might mean server side rendering all of the time, adding javascript only when absolutely necessary.

In regards to SPAs, I feel there must be a solution. Ember tries to do this with fastboot, allowing you to essentially have the flexibility of an SPA with the SEO and other advantages of serve side rendering, but I'll ignore that for now.

Is there a paradigm/language that allows you to write the same templates for both the server side and SPA view? I guess this would mean the view is the same, and the model and controller would be your choice.

I think it's natural to just do what you know, which is why you're probably seeing what you are.

This is immaturity, not a real need. The people who make the most money in tech aren't the ones with the most buzzwords on their resumes.

I have seen people spin up massive map-reduce clusters to perform something AWK could do in half the time. And I have seen spend enormous amounts of time training neural nets for something a simple SVM could do. In both cases (and most similar cases I have seen) a lot of needless time and energy was expended.

But why? This is my hypothesis anyways...

There seems to be two primary pools of people who feed into the software engineering. Those from engineering disciplines and those from research disciplines. Engineering schools can be fairly cut throat; limiting the number of candidates who can move on at various stages, or designing courses to reduce the number of enrolled students. Those from research disciplines are required to create novel ideas which are worth publishing just to graduate their programs. And those that actually like academia believe (and rightfully so) that creating new ideas is the way towards notoriety and success.

That means a lot of the craftsmen who would prefer making tiny refinements on well understood paradigms are simply weeded out from the discipline before they can even enter it.

I have known very smart people who quit engineering in college because they could simply not stand being in a room of people where everybody wanted to be the "smartest in the room".

Of course... the incessant CV polishing in Silicon Valley probably doesn't help either.

https://zeit.co/blog/next