Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Overengineering can kill a product

mindtheproduct.com

Post reply on HN
Print view View original

51–60 of 211 posts

Re: Overengineering can kill a product

#51

Post by bluGill »

lbriner wrote: post #50

As others have said, it is unecessary complexity. Over-engineering is an ambiguous term. For example, premature optimisation is frequently mentioned as over-engineering but is it premature optimisation to use a map/dictionary instead of an array for key-based access? Nope. That is just correct. Is it over-engineering to know that if your product succeeds, you could end up with X-hundred objects which will use up all…

> use a map/dictionary instead of an array for key-based access? Nope. That is just correct.

Maybe, maybe not. While the map/dictionary is easier to use, if the number of items is small (which it often is) the array will be substantially faster because the CPU will pre-fetch the next element into the cache while you are still checking if the current one is the right key. Maybe - check with a cache aware profiler, and you need to design your array to be cache friendly.

Of course the above assumes your map/dictionary is on the hot path (seems unlikely), your code is too slow (probably subjectively, but in some real time applications this can be objectively measured), and you are writing in an efficient language. IF All of the above are true, only then should you go to the complexity of using an array to do the job of a map/dictionary.

Re: Overengineering can kill a product

#52

Post by mindvirus »

abraxas wrote: post #44
yosito wrote: post #32

Six months ago I left a company that was working on an overengineered product. Even worse than it being overengineered was that it was also under documented. Working on anything was a pain, because the CTO wanted everything to follow his well thought out, and frankly very cleverly engineered design patterns, but he couldn't clearly communicate what those patterns were. And the entire company amounted to transforming…

Being ins a similar role myself, how do I ensure that engineers stay happy working on the project that we're working on? I'm finding myself actually doing the opposite of the CTO you mentioned and pushing them towards adopting more off-the-shelf components instead of maintaining homegrown stuff but I think I'm causing a degree of upheaval by doing this. Their justifications for push back however, often smell of sunke…

I think the best thing you can do as a CTO is define the problems/goals and desired outcomes very clearly. Think of possible solutions if you can but don't share them, and give your team the problem to solve. And let them stumble a bit, because in the level of buy in and growth you'll get is more than worth it - it'll be their solution after all.

Re: Overengineering can kill a product

#53

Post by jcun4128 »

jcun4128 wrote: post #2

I'm wondering about this now I guess it is old fashioned now to use environment variables and bare EC2 servers, managing your own APIs and websockets/DB on same server as opposed to breaking everything out. You need to use cloud formation and "oh did you know there is an AWS service for that?" Then you are using 5 services instead of 2. This twelve factor app concept. Don't know when is the right time to do this/at w…

anyway I'm probably just being stubborn like back in the day jQuery all the way now it's react everywhere which I'm used to now

Re: Overengineering can kill a product

#54

Post by crate_barre »

abraxas wrote: post #44
yosito wrote: post #32

Six months ago I left a company that was working on an overengineered product. Even worse than it being overengineered was that it was also under documented. Working on anything was a pain, because the CTO wanted everything to follow his well thought out, and frankly very cleverly engineered design patterns, but he couldn't clearly communicate what those patterns were. And the entire company amounted to transforming…

Being ins a similar role myself, how do I ensure that engineers stay happy working on the project that we're working on? I'm finding myself actually doing the opposite of the CTO you mentioned and pushing them towards adopting more off-the-shelf components instead of maintaining homegrown stuff but I think I'm causing a degree of upheaval by doing this. Their justifications for push back however, often smell of sunke…

As someone who is miserable on a similar project, the best thing you can do is give ample breathing room for the labor. The codebase is stress inducing, so being empathetic on deadlines and and what may appear to be shifting attitudes (good days, bad days) means a lot. Give off days early, try to make sure people pace themselves.

Otherwise they’ll burnout and the resentment of the state of the codebase will be jet fuel to that brand new revolving door on your team.

Commiserate to some degree, ‘look, I know this shit sucks’. If they don’t know you empathize, they won’t trust you.

Re: Overengineering can kill a product

#55

Post by kwertyoowiyop »

abraxas wrote: post #44
yosito wrote: post #32

Six months ago I left a company that was working on an overengineered product. Even worse than it being overengineered was that it was also under documented. Working on anything was a pain, because the CTO wanted everything to follow his well thought out, and frankly very cleverly engineered design patterns, but he couldn't clearly communicate what those patterns were. And the entire company amounted to transforming…

Being ins a similar role myself, how do I ensure that engineers stay happy working on the project that we're working on? I'm finding myself actually doing the opposite of the CTO you mentioned and pushing them towards adopting more off-the-shelf components instead of maintaining homegrown stuff but I think I'm causing a degree of upheaval by doing this. Their justifications for push back however, often smell of sunke…

It’s a real problem, if you’ve hired ‘architects’ whose skill and passion is creating new systems, rather than connecting and modifying existing tech. Video games went through this shift about 20 years ago, and it was painful. Many architects ended up moving to the game engine companies, or to other tech areas than games.

Re: Overengineering can kill a product

#56

Post by lkrubner »

On a related note, see "Don’t waste $1 million on devops infrastructure that you’ll never need":

http://www.smashcompany.com/technology/high-availability-is-...

This is a true story about an entrepreneur who was suffering a bad case of perfectionism. Instead of showing his MVP to potential customers, he just kept investing in the software, chasing an almost impossible ideal of high availability

Re: Overengineering can kill a product

#57

Post by Dave3of5 »

Overengineering is a non technical problem but of course most tech companies only interview devs on their technical skills.

I've found is that once a dev has an idea of how something is suppose to work it's hard to get them to think in a different way about that thing. So when that thing doesn't work out for some reason they just keep adding exceptions and adding exceptions until you have the horribly over-engineered solution.

Re: Overengineering can kill a product

#58

Post by jaywalk »

jrochkind1 wrote: post #33
scottious wrote: post #27

The insidious thing about over engineering is that it's usually committed by very experienced engineers. Experienced engineers rarely under-engineer, that tends to be fixed very early in one's career. As we get more competent and read more books, we have the tendency to get enamored by new fancy abstractions. We get too clever and then we get in our own way. Best real-world example: I inherited a project that was an…

OP suggests there's a hump in the curve of experience vs over-engineering; more experience correlates to over-engineering until it gets to a certain point, at which even more experience leads to less over-engineering. Is it true? I don't know, I think it matches my... experience. Part of it is that the "experience" needed to avoid over-engineering is helped if it's not just engineering experience, but domain experien…

My own personal learning curve matches that graph almost perfectly. I'm probably not as close to the right side as I'd like to think I am, but I'm always trying to move in that direction.

Re: Overengineering can kill a product

#59

Post by Griffinsauce »

darkerside wrote: post #17
a_square_peg wrote: post #15

On the other hand, simplifying an already over-engineered product is almost next to impossible simply because many jobs depend on it. Maybe I'm cynical but I'm beginning to think that software complexity grows until it justifies all the head counts in the department.

This is the same phenomenon as the Peter Principle, where people Rose to the level of their incompetence. It sounds like a joke at first, but, of course they do. People get promoted when they excel. When you are in over your head, you stop getting promoted. Of course, over time, people grow into their roles, and regain their competence. Devs will build software until they can no longer do so because the codebase is l…

> Devs will build software until they can no longer do so because the codebase is larger than their collective abilities to manage.

I'm developing a principle of radical simplicity to attempt to combat this: always keep things absolutely as simple as they can be.

It is always easy to add complexity later, never to remove it. So the only conscientious choice you can make is to keep things as simple as possible while satisfying the requirements.

You should also critically evaluate requirements that introduce complexity.

Also, I don't think people mention this enough here: complexity == bugs.

Re: Overengineering can kill a product

#60

Post by somehnacct3757 »

I don't think you can diagnose over-engineering after the fact. Unless you were in the room, or have access to written specs from a meticulously documented team, you don't understand the conditions under which the code was written.

Maybe the company was betting at the time on an integrations marketplace, or a lot of partnerships, so a robust integration platform was built. Then the company moved on to another bet after only writing one or two integrations on the new platform. Nobody over-engineered here. Everything was built to spec. But three years down the line the new dev is going to get assigned a bug in one of the integrations and lament how the code was over-engineered. Hindsight is 20/20.

Lots of stories from the trenches including many in this thread make this mistake. The same goes for 'tech debt'. If you weren't there, you don't know.

Post reply on HN