The junior developer sees a pattern and thinks why not make this a bit more generic so it handles similar cases. Oh but these cases fit a larger pattern.. and eventually you'll have written yourself an entire framework. This is very valuable, for the experience. Not for the framework, which will never be used. But you should go through it. The mid-career developer knows this from experience and sticks to the minimum…
Personally this is not my experience. Plenty of seniors spitballing not useful solutions, premature optimization of sort etc... The mid-career is perfectly fine, give you are also good at refactoring if needed. Also, another thing that I'm not seeing stressed enough on this topic is: measure.
Ask HN: How to avoid over-engineering software design for future use cases?
231–240 of 258 posts
Re: Ask HN: How to avoid over-engineering software design for future use cases?
#232Modify the code when the use cases change. In most of the times, open-closed principle is a trap.
However, over-engineering is not only a technical problem. It's more of a problem on project / people / finance, here are some examples but it would definitely not limited to these:
1. The margin is big and the team is big, so we need to keep these people busy.
2. The current stakeholder will cover the budget these iterations until we finish these features. after that, the maintenance costs maybe partially falls on us, or there's will be no budget at all. The less problems in the future, the better.
3. The current budget is for functionality A and someone pays it. And we plan to implement another feature B that is similar to these, and the budget comes from our own. Better make the solution generic so we can use it almost for free.
4. The list goes on...
Better fix those root problems first.
Re: Ask HN: How to avoid over-engineering software design for future use cases?
#233It really depends on what you're doing. If you're writing code for a device that's going to hang out in the forest for 10 years with no updates, write just enough code to solve the problem and make it easy to test. Then test the fsck out of it. If you're writing code for a CRUD web service that you know will get rewritten in 10 months, write just enough code to solve the problem and make it easy to test. Then, test t…
I don't really want to disagree with anything in here, this all looks good. But I think it's silly to imagine that there's nothing you can do to anticipate changes and make your code more amenable to change.
This is where "make it easy to test" comes in.
And, sure, there are some things that are easy to anticipate. If you're processing CSV, you may get asked to process arbitrary delimited files next. But the point is 3 - 5 years out, you'll get asked for something the current architecture cannot support and a major effort will be required. You cannot anticipate those and thus should never try to.
Re: Ask HN: How to avoid over-engineering software design for future use cases?
#234It really depends on what you're doing. If you're writing code for a device that's going to hang out in the forest for 10 years with no updates, write just enough code to solve the problem and make it easy to test. Then test the fsck out of it. If you're writing code for a CRUD web service that you know will get rewritten in 10 months, write just enough code to solve the problem and make it easy to test. Then, test t…
What are your thoughts on the habit of wrapping virtually every recurring piece of code in its own method? In case the implementation has to be overridden? E.g.: def current_time Time.now end vs. just repeating Time.now inline within all of the calling methods?
When it happens, deal with it - odds are it won't. Expend the energy making the code base testable not pretending to be an insurance company ("In case sh..").
Re: Ask HN: How to avoid over-engineering software design for future use cases?
#235Earlier quoted context omitted.
> The prescription for oneself is pretty simple - just say no! It is relatively easy to catch yourself whenever you think about some improvement that's irrelevant to delivering so long as you have convinced yourself that you need to wholeheartedly focus on delivering. That's fairly true if you intend a monomaniacal focus on delivery and you are dealing with improvements that are truly irrelevant. If other concerns li…
Two points here. First, I think the primary trap is that developers think they can judge what code is hygienic. This is the evil twin of the notion that developers can judge what part of a system is slow without profiling. Developers regularly hold religious wars on basic matters of code style. While you may profit from 'hygiene' there be dragons - developers often consider as hygienically critical such tasks as refa…
If software engineering is engineering, they can. It may be a skill that is weaker and stronger between different engineers varying particularly by relevant experience, and it may require complex and context (including present development team) sensitive evaluations, but it is a real skill that exists.
> This is the evil twin of the notion that developers can judge what part of a system is slow without profiling.
Well, it's not, because there is no analog to the “without profiling” part. It's true that there is an analogous tendency of prejudging problem code, but certainly, at a minimum, hygiene issues can be discovered by experience of problems of development/maintenance stemming from, e.g., code duplication for shared functionality instead of use of shared abstractions.
> The secondary trap is that developers think they can amortize up-front development costs over a long period of time.
Note that when I said long term I don't necessarily mean a “long period of time” but “some period longer than the minimum development time of the present item”...
> Projects get the axe if they do not deliver.
Yes, they do, but plenty of real projects aren’t, especially at initiation, delivering actual value every iteration, just progressively refined demonstrations (this is particularly common on replacements that, for whatever reasons, can't go the strangler/ship of Theseus appproach, and on those projects the team often has an idea where things need to be for real delivery. It's quite possible for code that makes subsequent tasks more costly even though it saves time this iteration to delay real delivery.
> Everyone thinks they are the reasonable person with discerning taste who only refactors code as necessary to optimize total cost.
No, in my experience that's not even approximately true. Most developers I've encountered think that, in their current team environment, they individually have a natural tendency either to excessively favor direct solutions that produce ugly code with outsized downstream cost or to go a few steps too far with overengineering abstraction (and most of them also recognize that that that overall tendency is only on average, and that they also miss on the other side sometimes.)
My point is that when you get beyond the simplistic rejection of things which have no relevance to the task at hand (which isn’t a scenario that occurs all that much), evaluating what is the right balance is nontrivial.
Re: Ask HN: How to avoid over-engineering software design for future use cases?
#236It really depends on what you're doing. If you're writing code for a device that's going to hang out in the forest for 10 years with no updates, write just enough code to solve the problem and make it easy to test. Then test the fsck out of it. If you're writing code for a CRUD web service that you know will get rewritten in 10 months, write just enough code to solve the problem and make it easy to test. Then, test t…
Nonsense.
Of course you can make reasoned predictions about how your software will change or not in the future.
Some will be wrong, but if you have any sense many will be right enough to have been worthwhile to address up front, and the balance will be clearly positive.
Re: Ask HN: How to avoid over-engineering software design for future use cases?
#237Earlier quoted context omitted.
>> engineers go extreme in designing things/code for future cases which are not yet known >They're afraid. In many companies (think FAANG), engineers, especially senior engineers are incentivized/forced to show fancy design docs as part of the annual appraisal process. The more complicated the design, the more 'foresight', the better. If it sounds kind of ridiculous (TPS reports from Office Space anyone?) it is. But…
I think it's a bit more than a desire to show "fancy design docs" or metric gaming. I've seen a lot of engineers prioritize fixing as yet hypothetical future problems over problems that are burning them right now even when there is no need for design docs and no metrics to game. I think some problems are just seen as sexier than others.
I just wanted to shed a light on some unfortunate external pressures faced by a subset of engineers, which contributes to this problem. Didn't meant to come across as cynical.
Re: Ask HN: How to avoid over-engineering software design for future use cases?
#238Earlier quoted context omitted.
Two points here. First, I think the primary trap is that developers think they can judge what code is hygienic. This is the evil twin of the notion that developers can judge what part of a system is slow without profiling. Developers regularly hold religious wars on basic matters of code style. While you may profit from 'hygiene' there be dragons - developers often consider as hygienically critical such tasks as refa…
> First, I think the primary trap is that developers think they can judge what code is hygienic. If software engineering is engineering, they can. It may be a skill that is weaker and stronger between different engineers varying particularly by relevant experience, and it may require complex and context (including present development team) sensitive evaluations, but it is a real skill that exists. > This is the evil…
Therein lies the question :)
> Well, it's not, because there is no analog to the “without profiling” part. It's true that there is an analogous tendency of prejudging problem code, but certainly, at a minimum, hygiene issues can be discovered by experience of problems of development/maintenance stemming from, e.g., code duplication for shared functionality instead of use of shared abstractions.
Even in this case we are limited. I've seen many cases where someone correctly identifies a piece of code that is painful to maintain and then dives in only to multiply the problem. We are even better off in performance land because we can test our code immediately after we write it to verify we haven't left things worse.
> No, in my experience that's not even approximately true. Most developers I've encountered think that, in their current team environment, they individually have a natural tendency either to excessively favor direct solutions that produce ugly code with outsized downstream cost or to go a few steps too far with overengineering abstraction (and most of them also recognize that that that overall tendency is only on average, and that they also miss on the other side sometimes.)
I don't want to wade into the thicket on this one except to note that it's possible for someone to articulate something they may even believe on some superficial level without truly internalizing that belief and that IMO I observe this behavior a lot.
I don't want to invalidate your point outright because I think it's valuable and I even agree that it's rare to encounter a code base that would not benefit at all from 'code hygiene.' All models are wrong but I think an aggressive focus on delivering functionality is a useful one in most contexts. Put another way, a good but imperfect rule of thumb is to refactor only in order to gain something tangible for end-users.
Re: Ask HN: How to avoid over-engineering software design for future use cases?
#239I am a firm believer that the most important superpower a software engineer and their team should have is refactoring. By refactoring what I mean is continually revisiting the architecture of your code, identifying common functionality, better organisation of code, better abstractions. The right decisions for your codebase change as it evolves and so you need to keep reexamining these (implicit) decisions. Continuous…
Code should be treated as expendable and transient, your clever implementation might be considered deprecated in a month due to changes in requirements/design.
Unfortunately it's not easy to convey this idea as a team lead. People get "attached" to their code and the idea of refactoring is viewed as mostly unnecessary and dangerous (I wrote the code, I'm not gonna rewrite it)
Re: Ask HN: How to avoid over-engineering software design for future use cases?
#240Earlier quoted context omitted.
> In many companies (think FAANG) > The more complicated the design, the more 'foresight', the better. Certainly not in Amazon. Surely there can be exceptions but in general the company has a culture of simplifying stuff anywhere is possible.
>simplifying stuff anywhere is possible https://raw.githubusercontent.com/aws-samples/aws-refarch-wo... Ironically, that's how you are supposed to run Wordpress on AWS.