Ask HN: Is Accidental Complexity Growing Faster Than Essential Complexity?

Many companies have too many engineers (and some not enough). This is mainly due to #engineers = f1(revenue || vc money) while it should be #engineers = f2(problem). Mostly because our industry has no clue about f2 and so falls back to f1.

But this also works with business thinking, e.g. CEO thinking

techbudget = 0.1 * revenue

With the law of the used budget (all people always use up their budget of fear to not get the budget next year when they might need it), CTOs use the whole budget

#engineers = techbudget/salary

so f1 is often

#engineers = f1(revenue) = (revenue * 0.X - hosting - laptops - licenses)/ salary

and with salary >> hosting, salary >> laptops, licenses -> 0 due to open source usage,

f1(revenue) = revenue * 0.X / engineer salary

(the 0.X is determined by VC experience/push, negotiation between CEO and CTO, how much of a tech company the CEO sees his company or from his previous experience, most probably on a wholly different tech business model).

with no need to understand the tech challenge at hand.

For startups and high margin tech business, there is often a large tech budget, so many engineers are hired (also other forces like capability building, war of talents etc. - some startup CEO tell me their VC said they need to hire 100 engineers until the end of the year - this is without knowing anything about the tech problem at hand).

Coming to your point:

If there are too many engineers for the essential complexity they create accidental complexity.

Edit: I just checked the meaning of 'died down' and it means 'to reduce in strength' not 'to become literally dead'. So, my comment isn't really a correct response to the parent but I keep it here.

Do you really think all those concepts died? IMO, they aren't dead, they have been assimilated. We learned from OO and it's now a tool you can use almost everywhere when you need it. But we also learned to not try to model everything in an OO paradigm.

You can say the same thing for your other examples. You have a phase where the concept/techno is being seen as the silver bullet that will save us all. That brings a lot of adoption. People do mistakes. We (hopefully) learn from them. The hype fades away, concepts are consolidated and assimilated in stable and mainstream tools.

But that takes time. And I'm not sure how things could work differently. Don't we need the hyper-hype phase to experiment and learn about the concept/tech?

I really can't address the issue of overuse of microservices in a line of business app type project, but I want to push back about little about "SPAs" being "accidental complexity" here. I imagine by "SPA" you're referring to the use of a JS framework like AngularJS, Backbone, EmberJS, React(and some type of models), etc.

While I agree that sometimes there is no need for these in a simple CRUD app, I'd also caution that depending on your basic needs (frontend validation, autosave, keyboard shortcuts, etc) and frequently, on more complex client requests (like for fast JS-driven form navigation that requires some sort of frontend routing, or to save to some industry-specific file format, and such), you can easily end up with a big spaghetti ball of JQuery if you're not careful.

I've seen these balls of tangled JS all too often in line of business apps, and they're a nightmare to maintain or modify. The point of using a framework is to provide some sort of organization for frontend JS, and to provide common features.

If your team has the maturity to organize their frontend JS into neat MVC/MVP apps without the use of a JS framework, great. But in my experience, few "full-stack" developers are comfortable enough and/or care enough about frontend JS to do this. Thus the use of frameworks.

If you need a very minimal framework, BackboneJS is both tiny and simple to use. Even Backbone can save simple apps like this a lot of grief.

The alternative to something like this is to stick to a full-stack framework like Ruby or Django and stay strictly in that environment/best practices. But even there you're bringing a lot of complexity into the picture, or sometimes you still end up needing some sort of organizing principle or framework for the frontend JS.

Well, a theory is that Agile is responsible for both. While its short devlopment cycles help deal with essential complexity (you understand better the customer's problem because you have more interaction with them), it also implies incremental coding which often increases the accidental complexity.

As an example:

After a short development cycle, you created a database schema where each Client class has a single address, and because it was done quickly, the address fields are in the same Client table.

In a second development cycle, your customer tells you that their clients may have one or more address and one of them is the primary. The quickest path to delivery is to keep the address in the Client table as the primary address, and add a secondary address table.

While short development cycles help in not screwing up large requirements, it also impairs our ability to see the larger picture, and cause us to take decisions that we need to undo afterwards.

I believe it's a little bit of "column A" and a little bit of "column B".

On one hand it can feel maddening that a generation (or two) of young IT professionals have seemingly gone full-circle and tried all kinds of techniques only to end up leaning on tried-and-true technologies of the past (think: NoSQL vs ACID compliant relational DB's, Ruby/Python vs Elixir/Erlang, Flash/Flex - XML based design vs React/Javascript/HTML5/Canvas, Dynamic vs Strict typing, etc).

There has been a lot of re-inventing.

However, I think this had to happen. Many great ideas of the past were amazing ideas but they simply were before their time, not quite implemented well enough, we're too heavy/complex for the hardware at the time, and so-on.

Software has been moving too fast for the collective mind to keep up.

Maybe, in an ideal world, we would have methodically built up on the best of software from the 60's without wasting time and effort experimenting with new ideas. Honestly, that might have been awesome for productivity and the advancement of humanity as a whole. That's not how we seem to function as people, though. I can't help but think of "space travel" here (it sure didn't carry on exponentially from the moon landing ~50 years ago).

In my mind I believe this always happens with technology of any kind. It may be why we're discovering that ancient civilizations probably had much more interesting technology than we initially imagined. They simply has certain cycles where they "really hit a sweet spot" and other periods where it all became contrived.

On that note, if anyone has any suggestions for books about this subject please let me know. I know it's not an original thought and I'd bet there's been some interesting research.