Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: How to avoid over-engineering software design for future use cases?

news.ycombinator.com

Post reply on HN
Print view View original

41–50 of 258 posts

Re: Ask HN: How to avoid over-engineering software design for future use cases?

#41

Post by chasd00 »

Knowing the point where you pass good design and start over-engineering is an art developed through wisdom and experience. Sort of like the art of making good LOE estimates given incomplete requirements and an unknown team.

I would look to senior people who have boots-on-the-ground experience delivering and maintaining projects/products, ask them if they think you're overdoing it.

edit: I want to add, get input from non-engineers too. Ask them "in your experience, how far have projects diverged from the initial requirements/purpose? I'm trying to plan for the future but not over-complicate things"

Re: Ask HN: How to avoid over-engineering software design for future use cases?

#42

Post by dgb23 »

kls wrote: post #21
dgb23 wrote: post #11

Disclaimer: I have never worked in large teams but this problem arises everywhere with the caveat that in smaller teams and solo development the coordination requirements are much simpler. That said, I think there are two steps in designing abstractions: The first is to split up and isolate both code and data into small, simple pieces. These can easily be non-DRY (structurally) and often are. You'll have cases where…

I would agree with the parent here, try to not go into abstractions until they are apparent but there are some pretty solid patterns that have worked thru the years and still hold up today. The first being pub-sub, this can be an event system or and observer pattern but it is good to have loose coupling of when something happens it can loosely tell it out into the ether and not care about if anyone is listening. The…

I fully agree with the first. You usually need some designed way of handling communication and/or IO in a sufficiently large application. Whether these are event queues, channels, commands, pipelines or w/e depends. But thinking about this up-front is the key here.

I would describe the second as a constraint rather than an abstraction. And I fully agree with this. The most obvious and probably most talked about benefit of FP would be avoiding state management complexities.

The third one is neat. But I would say we're already in danger territory here. Just writing a function and naming it properly is the minimal step required to make refactoring into a strategy or plugin pattern later fairly straightforward.

Re: Ask HN: How to avoid over-engineering software design for future use cases?

#43

Post by mishaker »

This is a typical case I am struggling with during my whole career in software. It is very hard to fight against because it is counter-intuitive. Saying that "let's put aside all those designs and let's do a minimalistic version to satisfy current needs and then we move from there" is always less attractive than "let's do a great architecture which can support our 'exponential growth'".

In my experience following categories of people tend to fall into the trap easily (and it is really hard to convince them):

- Corporate people.

- Business school people.

The following can amplify it:

- Fund raising... you were successful in raising money so that means there is a market and people love your product ... well guess what? you are wrong.

If you are in such a situation, debating is not useful. Changing this mindset is only possible when you hit the wall at least once in your life. BUT ... you can accelerate the process of realizing it (before it is too late). Try to find a way to make people realize that all those bells and whistles and designs were not necessary cause nobody actually asked for and we are changing it anyway now.

Re: Ask HN: How to avoid over-engineering software design for future use cases?

#44

Post by econcon »

By having a time limit. When I was new and over excited young guy I started with the assumption that we'll be working on it for a really long time but it rarely proved true.

Truth is most projects will be shelved before even they are complete.

You'll be taken off some projects because of financial or political BS.

So start with the assumption, there is limited time to deliver - now ask yourself - how can I maximize the effectiveness of that time? How can I do what truly matters without going into the rabbit hole of optimizations and using the best thing possible everywhere.

And once you've done that, you can always come back and improve the thing if the project is still around but most likely it will not be. Either you'd have switched the company or company would have switched you or the project has switched the company.

Re: Ask HN: How to avoid over-engineering software design for future use cases?

#45

Post by daniel-levin »

32 comments so far and no mention of the word budget. There's a great analogy between software engineering and construction. Does your organization build skyscrapers and gorge-spanning bridges? Or does it build driveways and swimming pools? Commercially developed software consumes capital to get something in return. Are the people spending capital budgeting for a driveway or for a skyscraper? Must it be done this month or in two years? Sure, those are false dichotomies, but they illustrate the point: it is desirable for the bosses to clearly define engineering spend.

Over-engineering can be avoided by carefully sticking to a budget.

For a lot of developers, there's a trade-off between rationality (in the sense of ROI) and feeling good. It doesn't feel good to make every engineering decision against a budget. My dopamine levels [0] skyrocket when I visualize making some component generic, or future proof. I come crashing back to earth when I realize the budget is for a driveway. Budgeting earns the bosses / customer / capital a better return. Engineering for future use cases or making things generic (or the mere anticipation thereof) is an easy way to get a massive hit of neurotransmitter that makes you feel good.

[0] Not a neuroscientist. But I think it's useful to label that spike of feel good and motivation. It may have nothing to do with dopamine.

Re: Ask HN: How to avoid over-engineering software design for future use cases?

#47

Post by tmaly »

I personally like the simplicity of designing for change. Simple rules like TDD help you to think about the design up front.

I was talking to a new guy that just joined my team yesterday on this subject. You really cannot predict the future.

You could also look at it from one other angle. If you are only building the bare minimum to satisfy the requirements, that is a lot less code you are writing. If you need to replace the system, that is a lot less work to go back and rework.

Re: Ask HN: How to avoid over-engineering software design for future use cases?

#48

Post by Nursie »

daniel-levin wrote: post #45

32 comments so far and no mention of the word budget. There's a great analogy between software engineering and construction. Does your organization build skyscrapers and gorge-spanning bridges? Or does it build driveways and swimming pools? Commercially developed software consumes capital to get something in return. Are the people spending capital budgeting for a driveway or for a skyscraper? Must it be done this mon…

> My dopamine levels [0] skyrocket when I visualize making some component generic, or future proof.

I think I've successfully rewired that impulse in myself now, at least partially. I do get that feeling from imagining great systems, but I also get that dopamine hit from deleting old stuff and simplifying things as much as possible. We can ditch compatibility for API v8? Awesome, I just made my code smaller and we can scrap that entire abstraction layer! The service just lost 20% of its LoC :)

Re: Ask HN: How to avoid over-engineering software design for future use cases?

#50

Post by mason55 »

JoeAltmaier wrote: post #31
mmxmb wrote: post #10

The chapter The Second-System Effect form The Mythical Man-Month book (Brooks, Jr. Frederick P.) talks about this problem: “An architect's first work is apt to be spare and clean. He knows he doesn't know what he's doing, so he does it carefully and with great restraint. As he designs the first work, frill after frill and embellishment after embellishment occur to him. These get stored away to be used "next time." So…

Yet 'spare and clean' can mean breaking the problem down into primitives. And primitives can be reusable. I agree, contriving baroque features and APIs are usually wasted time. But planning simple basic operations that others are built upon is good sense. When possible.

Yeah we call them "building blocks." Instead of building specific individual features we create building blocks that we can use to implement features. The key difference is that by exposing the building blocks to customers they can use the building blocks in ways that we didn't imagine.

The most important reason for us to do this is so that our customers can always accomplish what they need, even if it's somewhat manual or painful. By exposing our building blocks we never have an emergency where we need to implement the feature "copy foo to bar so that customer X can accomplish important thing Y." Instead we can say "you're able to do that manually and we'll talk about a feature that will help you automate if the manual process is too painful."

Sometimes you just need to write some straight business logic, don't get me wrong, and it takes experience to know when to create a new concept and when to just bang out some code. But exposing primitives or building blocks reduces the amount of "critical" development substantially.

Post reply on HN