Overengineering can kill a product • Hacker News Board

Re: Overengineering can kill a product

#121

Post by hinkley » Wed, Nov 24, 2021, 4:12 PM UTC

yosito wrote: post #32

Six months ago I left a company that was working on an overengineered product. Even worse than it being overengineered was that it was also under documented. Working on anything was a pain, because the CTO wanted everything to follow his well thought out, and frankly very cleverly engineered design patterns, but he couldn't clearly communicate what those patterns were. And the entire company amounted to transforming…

> they were struggling to find engineers that the CTO thought were smart enough.

If you have to find the smartest people to keep the wheels on, you’ve already lost.

Disdain for the bell curve is the fastest way to get overengineering. Very few things have to be rocket science to create a good company. Everything else should be dead simple so that anyone can work on it. That also means you have to compartmentalize the hard bits so they don’t leak across the entire codebase.

But some people get bored with mundane code and will make things interesting by replacing a boring task with an exciting one. It’s part of the Curse of Knowledge. You can’t run a company like an action movie. Everyone gets burnt out.

Re: Overengineering can kill a product

#122

Post by baby » Wed, Nov 24, 2021, 4:13 PM UTC

wheelerof4te wrote: post #87

I often find myself prefering languages with few abstractions even when some more complex language might be "better". Reason is that with "bare" languages like C, you can choose your own design more freely. Case in point: Serenity OS. Author of that OS replicated entire libc and still refuses to use C++'s STL. He created his own abstractions that he feels confident using. That is true engineering.

You should check golang, it’s much more safe than C and has very few ways of doing the same thing. It makes every codebase look like simple code. It’s always a pleasure to read or refactor. That’s the only language that managed to do that imo.

Re: Overengineering can kill a product

#123

Post by svnt » Wed, Nov 24, 2021, 4:13 PM UTC

alex_sf wrote: post #108

what_is_orcas wrote: post #93

Earlier quoted context omitted.

I politely disagree. I'm currently walking away from a code-base (and the company who let it be built) that was obviously built by someone who's mental model of how the code works was incorrect. At some very basic levels, it under-performs compared to literally every other example of this sort of code I've ever seen (and it's extremely common), which results in a subpar user-experience and an EC2 instance way bigger…

Sounds like there are multiple employees. Does the product generate the revenue for their paychecks? If so, it's working correctly.

I understand the sentiment but there are so many situations where this is not true.

All it takes is a good enough sales and marketing team, or the right executive relationships, especially if the team being “supported” is small.

You can say the company or division is presently functioning, but the time constant on the product is different, and the coupling is too loose to say anything else with accuracy.

Re: Overengineering can kill a product

#124

Post by GuB-42 » Wed, Nov 24, 2021, 4:14 PM UTC

While "overengineering" is usually defined as making things too complex, I think it is a bit misleading, that's because simplifying, removing or making things cheaper is also engineering. In fact there are many well paid engineers who work full time simplifying things.

As much as I hate the cult of Elon Musk, I have to admit that's one aspect of engineering he really gets, and I suggest you listen to his interviews with Tim Dodd / Everyday Astronaut, where he gets technical, trust me, it is not the usual bullshit.

All that to say that "overengineering" is bad doesn't mean you shouldn't have a lot of engineers working hard on a problem. In fact, a lot of "ovengineering" is the result of not enough engineering. They picked a (complex) solution without thinking instead of really studying the problem.

Re: Overengineering can kill a product

#125

Post by throw10920 » Wed, Nov 24, 2021, 4:15 PM UTC

nisa wrote: post #3

Complexity kills your product - Overengineering is just one instance of complexity - Technical Debt like having state and data all over the place is another one - I quit my last job and I happily blame this article for convincing me to quit: https://itnext.io/the-origin-of-complexity-8ecb39130fc - coordination causes complexity and this killed me - we had everything not once but twice or more in different places - ju…

> Complexity kills your product

This is like saying that water is lethal - both are true in the sense that some x (water, complexity) is ok, but too much is bad.

Complexity is an unfortunate (but necessary) side-effect of (1) adding features and (2) optimizing for performance, both of which are critical for building a product.

If you try to remove all complexity from your product, you won't have a product. Instead, you have to try to minimize complexity while still delivering the same set of features and level of performance.

Re: Overengineering can kill a product

#126

Post by PragmaticPulp » Wed, Nov 24, 2021, 4:17 PM UTC

somehnacct3757 wrote: post #60

I don't think you can diagnose over-engineering after the fact. Unless you were in the room, or have access to written specs from a meticulously documented team, you don't understand the conditions under which the code was written. Maybe the company was betting at the time on an integrations marketplace, or a lot of partnerships, so a robust integration platform was built. Then the company moved on to another bet aft…

> I don't think you can diagnose over-engineering after the fact. Unless you were in the room, or have access to written specs from a meticulously documented team, you don't understand the conditions under which the code was written.

The specs and the decisions surrounding the code are usually part of the over-engineering.

> Everything was built to spec.

Even with reasonable specs, I've worked with, and later managed, engineers who had a strong drive toward over-engineering solutions. I've also done it myself, and I suspect most people reading this have done the same.

The problem is that many of us engineers became developers because we enjoy building things and we enjoy solving complex problems. It can be exciting to imagine a particularly complex version of a problem and how elegant and fun a complex solution would be.

It can be as simple as a programmer who tries to write their own CMS when the spec could be solved with a simple blogging platform. If you're an ambitious programmer, writing a CMS is fun! Installing Wordpress is blech.

I didn't truly understand the ubiquity of the problem until managing ambitious engineers who wanted to do complex things. At times it felt like 1/4 of my job was just asking "If we wanted to simplify this solution, what would that look like?" Repeat for a few rounds and eventually the team has shed a lot of unnecessary complexity without sacrificing the spec at all.

Re: Overengineering can kill a product

#127

Post by baby » Wed, Nov 24, 2021, 4:17 PM UTC

typicalbender wrote: post #82

somehnacct3757 wrote: post #60

I don't think you can diagnose over-engineering after the fact. Unless you were in the room, or have access to written specs from a meticulously documented team, you don't understand the conditions under which the code was written. Maybe the company was betting at the time on an integrations marketplace, or a lot of partnerships, so a robust integration platform was built. Then the company moved on to another bet aft…

This is why commit messages are so important. Documentation that is external to the code its referencing will always drift but a commit message is a snapshot in time that is attached to the thing it is referring to. In my experience not enough people take commit messages seriously and just basically give a summary of what changed not why it was changed.

Commit messages get lost too easily. It’s the worse way to document reasoning around the code. There’s many proposals to document code outside of code and none of them has ever worked. Just comment your code.

Re: Overengineering can kill a product

#128

Post by yuriy-yarosh » Wed, Nov 24, 2021, 4:18 PM UTC

Seems too oversimplified and biased.

Probably missing few core points:

1. Without good development practices due to lack of Experience or plain old Rational Thought the only possible outcome is Operational Deficiency.

Every Best Practice is context-dependent, thus having Deficient Resources makes them inapplicable in certain cases. Some teams can really suck with the same tech stack when others flourishing using it.

2. Basic organizational anti-patterns, like Mushroom Management, and broken retrospective lead to rediculous outcomes.

Even plain old Micro-services and Micro-frontends can be a basis of Stovepiping and applying Mushroom Management. Usually, again, due to Lack of Competence and Sheer Hubris.

3. “Premature Optimization” only used in context of over-engineering by those who didn’t read the book, but use Halo-effect cognitive bias to project compensated qualities onto the term itself. There are a lot of Psychological Compensational Needs under the hood.

It’s like “Why Agile has nothing to do with Discipline ?” or “Why senior developers turning the project into a sandbox due to the lack of self-fulfillment ?” or “Why most of the MVP’s lack Concise and Validated Definition of Viability ?”

Complex doesn’t mean Hard or Expensive. Simple doesn’t mean Easy or Cheap.

Too often “over-engineering” is just an organizational and psychological issue and not an Engineering one.

Stop operating on Feelings. Six Sense of the Fifth Body Anchor Point is not a reliable Key Performance Indicator.

Re: Overengineering can kill a product

#129

Post by what_is_orcas » Wed, Nov 24, 2021, 4:19 PM UTC

marcosdumay wrote: post #103

what_is_orcas wrote: post #100

Earlier quoted context omitted.

I don't disagree, but I think that the reason that this fits here is because the poorly engineered software is unnecessarily complex (not in the "someday we might need this" way, but in ways that I'm having a hard time articulating without giving away the product because I know my coworkers are on HN). In fact, it was the ridiculous complexity that drove me to discover the disparities between the original developer's…

You see, overengineering is exclusively of the "some day we might need this" kind of complexity. Anything else is a different problem, very likely with very different causes, even if the consequences are similar, so it's not useful to analyze together.

There is also some of this.

Some of it makes sense when the reasons were explained (e.g. "we figured this would make it easier for [business unit] to do [thing] which turned out not to be something [business unit] actually has interest in doing").

Other examples are purely of this sort (e.g. "we'll be glad we have this in place when we finally hit [x] users or enter [market]").

Re: Overengineering can kill a product

#130

Post by inerte » Wed, Nov 24, 2021, 4:19 PM UTC

gampleman wrote: post #66

One thing that bugs me is the notion that "Software rewrites are something you should never do", which is a mantra so often repeated that it has acquired the status of self-evident truth, despite the only evidence being (usually) presented is an example of a web browser from 20 years ago! (Which incidentally spawned Mozilla, so not exactly a complete loss; especially from the POV of society rather than shareholders,…

Rewrites are also sometimes very good for your career. A friend working at Google for about ~1.5 years said a new Sr. Director of Engineering joined and their first decision was to rewrite the project, and it would take 2 years.

So now the Director has job security for about 2 years, gets the big launch near a promotion cycle they have a small chance of being considered for promo, and gets to blame the predecessor for all the problems.