Another point that goes along with this: good engineering is not premature optimization[1].
Perfection is not over-engineering
31–40 of 132 posts
Re: Perfection is not over-engineering
#32I wouldn't say that "over-engineering means solving the wrong problem". It's possible that the idea is basically correct, but people are directing effort towards optimizing for constraints that don't really exist or can be dealt with once a better picture is in place, whether it's the mythical PMF or just "we now understand what the users want, let's build that". The worst clusterfuck I've ever worked on was a web ap…
> but the team was spending time building an absurd Rube-Goldberg contraption of microservices This is literally the example that I use, the most common case of over-engineering, having more microservices than team members. Microservices are the right solution for certain problems, but those were not problems they had. Some anecdotal evidence. I worked in many of these places, and the most common tell of over-enginee…
> absurd Rube-Goldberg contraption of microservices
Such a lovely analogy. FYI it’s mine now.
Re: Perfection is not over-engineering
#33Perfection is not "perfectionism". Former is provable given set of constraint. Later is anxiety due to fear of criticism.
"We don't want to build the perfect solution." Can both mean "no need to cover use cases we don't care about" and "no need to deal with divizion by zero in a product, where a number is divided by the user entered number".
Re: Perfection is not over-engineering
#34I'm all for pushing back against "let's not make perfect the enemy of good." I hear that all the time in reference to software that is usually quite bad, and often a little evil to boot--nothing good anywhere in sight except some poor schmuck trying to find a way to make their job something worth taking pride in. I'm not sure I agree that systems are products though. The product mindset is toxic. It means that you've…
You've definitely used non-toxic and thoughtful products or you wouldn't be making such a distinction. You are throwing the baby out with the bathwater by retreating to "tools". If you want to stop the enshittification, this is not the way. You're just opening yourself up to new scams. i.e. AI tools, political agendas, etc. Better products are just better. Demand them.
When I think of software that is minimally tainted by productism, these come to mind:
- linux
- nix
- nushell
- helix
Are these products... at all? Do the open me up to new scams? I don't think so.
I'm not in a position to demand anything from the people who make these things. If they were subject to demands, their craft would be tainted by compromises made in acquiescence to those demands, and I'd probably be less enthusiastic about their software (because presumably, my tastes don't align with whichever others are also in a position to be making demands).
Supply and demand are well and good if what you're after is barley. But when you compare what there's demand for with what's being supplied re: software, there appears to be no correlation.
We gotta stop selling picks and shovels and start learning to be miners who have good taste in picks and shovels and the ability to make and remake our tools as needed. The disconnect is creating a hell for our users.
Re: Perfection is not over-engineering
#35There is no such thing as perfect.
Anyone saying "don't make perfection the enemy of good" is using a thought terminating cliché, avoiding the conversation of what is good enough. Worse, it is often used by people to drag quality down. If you're creating the "minimum viable product" you usually create a product that isn't working.
But, perfection doesn't exist. Most solutions in the world have no global optima. There are always tradeoffs. You must choose. You must argue with your peers to figure out that tradeoff. Perfection has infinite depth in detail. You must optimize. You just learn the unknown unknowns in an every moving landscape.
But you should still chase perfection. Like you chase a utopia. Like you chase your dreams. There's always something to improve on. Chasing perfection while knowing it can't exist means you will continue to search for the flaws. It means you will continue to improve. "Over engineer", because that's just engineering. Make things actually work, while recognizing they're always broken somewhere. Don't get offended when someone points out a flaw, you already know it's not perfect, so figure out if it is a tradeoff or can be fixed. Just keep improving things, because otherwise they keep getting worse
Re: Perfection is not over-engineering
#36I'm all for pushing back against "let's not make perfect the enemy of good." I hear that all the time in reference to software that is usually quite bad, and often a little evil to boot--nothing good anywhere in sight except some poor schmuck trying to find a way to make their job something worth taking pride in. I'm not sure I agree that systems are products though. The product mindset is toxic. It means that you've…
I'm also wondering if "perfect" and "good enough" are not really as important now vs. when a team of software engineers had to spend sprints implementing features. The rate of iteration is faster now, the rate of regenerating entire code bases is days. We can perhaps over engineer /more/ today than before.
The real challenge is reigning in the size and complexity of the codebase if you're using AI to generate it. I have a bunch of skills (YAGNI / KISS inspired) but it still requires a significant amount of human effort. Knocked down about 9000 lines of cruft this month and I know there's another 10k in there sitting around.
Re: Perfection is not over-engineering
#37I wouldn't say that "over-engineering means solving the wrong problem". It's possible that the idea is basically correct, but people are directing effort towards optimizing for constraints that don't really exist or can be dealt with once a better picture is in place, whether it's the mythical PMF or just "we now understand what the users want, let's build that". The worst clusterfuck I've ever worked on was a web ap…
While I agree with you, a large number of people use the term to mean that something was engineered to a complexity they have difficulty understanding. Abstraction is also treated as a dirty word, with people focusing on only one interpretation of the word.
While I don't actually agree with OP, I do agree with their sentiment. I've seen people say something is "over engineered" when there's an elegant design. Elegance isn't over engineering, it is solving problems effectively. It's something we should chase! Elegance is solving the right problem, which usually people are having a hard time seeing. (It's not always easy)
If we constantly let people drag quality down then we get into this frustrating world where everything is constantly half broken.
Re: Perfection is not over-engineering
#38The idea sounds related to the distinction between essential and accidental complexity.
https://ferd.ca/complexity-has-to-live-somewhere.html
I really like the article above in that complexity does have to live somewhere. If you have a hard problem to solve, there is a minimum level of complexity (essential complexity) that can't be further reduced, only moved from one place to another.
And yet we often add accidental complexity by making bad choices (thinking of you, microservices). If your choice is bad enough, you get all the disadvantages (challenges with distributed systems, complicated debugging), but without any of its advantages (a distributed monolith doesn't get you any deployment or team independence).
Telling one from the other, well, is something I'd like to learn and that seems to require both experience and some sort of intuition.
Re: Perfection is not over-engineering
#39I don't share the definition of over engineering. As engineering is the act of solving technical problems, over engineering is about putting too much engineering effort on aspects, features or products that don't have a linear payoff to the budget spent. E.g. I worked in a company that was obsessed with unit test coverage metrics and the effort of maintaining the test suite was considerably biting in the ability to m…
This is a good way of putting it. People who advocate for over-engineering don't consider cost-benefit vs not doing it. This is where FMEA analysis comes in when developing products, you assign values for severity, occurrence, and detection to determine which potential issues need extra effort implemented, but just as important is identifying the potential issues that aren't worth putting effort into to prevent over-engineering.
I've recently had to deal with a problem where the parent company has dictated 100% production testing for a spec that we meet for the spec sheet but customers practically don't ever need. But since the parent company test their products this way they say we have to. I've been very vocally against this because it has delayed us shipping products that are otherwise ready, prevented starting new products, and added a lot of production cost when our RMA rate without the test is something on the order of 0.01% and most of those are due to installation errors, not true failures.
Re: Perfection is not over-engineering
#40Hang on. Let me overthink this.
Let me do that for you. I think perfection is subjectively met at the end of one's consciousness about a problem. A higher being will see the problem more deeply, and its perfect solution could therefore be more perfect than that of a lesser being.
Imagine that one has to implement a function to multiply two large integers. A junior programmer will likely use the default multiplication operator. A mid-level programmer will add a couple of strategies with a weak heuristic for routing, and perhaps with basic caching. A senior will sample the incoming requests to identify which strategy might empirically be good, and ensure that it is being hit, with optimal fallbacks. A superintelligent AI might design custom kernels, and design a custom model for routing. A god will... we don't know what a god will do. Perfection, like beauty, is in the eyes of the beholder.