Live data from Hacker News

Ask HN: How to avoid over-engineering software design for future use cases?

news.ycombinator.com

21–30 of 258 posts

Re: Ask HN: How to avoid over-engineering software design for future use cases?

#21
post #11

Disclaimer: I have never worked in large teams but this problem arises everywhere with the caveat that in smaller teams and solo development the coordination requirements are much simpler. That said, I think there are two steps in designing abstractions: The first is to split up and isolate both code and data into small, simple pieces. These can easily be non-DRY (structurally) and often are. You'll have cases where…

I would agree with the parent here, try to not go into abstractions until they are apparent but there are some pretty solid patterns that have worked thru the years and still hold up today.

The first being pub-sub, this can be an event system or and observer pattern but it is good to have loose coupling of when something happens it can loosely tell it out into the ether and not care about if anyone is listening.

The second being sole responsibility of change, this does not have to be something as formal as a Redux or some other library if you are using a back-end language but there should be one function that changes one segment of data and that is the one function that owns the writing of that data.

The third is more optional but in certain places it really shines and has become kind of a lost art and that is plug-in style interfaces. Say you have a portion of you app that parses text files and stores them in a common data structure and you know that there will be future text file formats but you do not know what they will be. A plugin architecture is a natural fit here, where you define the interface and the data storage side, then you only need to implement a plugin for the parser side for each new file format. I always write these as true plugins where you can drop a new one in a plugin folder and the application picks it up, no recompiling no configuration, no restarting, just a black box that is self contained.

Re: Ask HN: How to avoid over-engineering software design for future use cases?

#23
post #18

> engineers go extreme in designing things/code for future cases which are not yet known They're afraid. Fear: If I don't plan for all these use cases, they will be impossible! I will look foolish for not anticipating them. So let's give into that fear and over-architect just to be safe. A bit of the 'condom' argument applies: better to have it and not need it than to need it and not have it. But the reality is that…

Also: have a framework in place, which supports worry-free refactoring.

Comprehensive Unit/Integration tests, a robust type-system, pick whatever suits your style.

It's a lot easier to refactor stuff when you don't have to worry about breaking something hard to debug with a big code change.

Re: Ask HN: How to avoid over-engineering software design for future use cases?

#24
post #14

When I was young and fresh, I wanted to cater for all sorts of future possibilities at every turn. But what if things change this way? What if things change that way? I came to realise that when things change, unless the design change is trivial (allow use of database Y as well as Z, etc) then you probably haven't anticipated the way in which it is going to change anyway. Better to have clean, straightforward code th…

Less is more. The simpler your code is, the easier it usually is to change to meet new requirements.

If you need to push every variable through 15 layers of interfaces and factories and generators, you'll have a bad time.

Re: Ask HN: How to avoid over-engineering software design for future use cases?

#25
I like to build a forward looking document for a team/groups software which extends out to cover the companies goals + N months/years. The goal of the document is to help define the core components, their role, and integration points such that everyone can reason about future looking tradeoffs. Teams can then use the document to see how their specific roadmap fits into the broader picture. This also helps frame use cases terms of current, goals, and dreams. If you're building for a use case beyond the team's dream features then you're probably over engineering the solution.

In practice goals start to move after horizon/2. And a new document needs to be created to capture where the business is doubling down and where it is trimming.

Re: Ask HN: How to avoid over-engineering software design for future use cases?

#26
I don't mean to be controversial but I think planning like that shows lack of experience and more than likely a problem with how work gets done. The developers might feel they have no chance to develop the software further after the first release. That means you have to capture all scenarios straight away.

Re: Ask HN: How to avoid over-engineering software design for future use cases?

#27
Don't design for future use cases unless they are known in detail (in which case they're not really future use cases anymore). Things you don't know, will change. Your extra effort may actually hurt you in the future.

Better is to design for change. Keep everything modular. Keep your concerns separated. When something needs to change, you can just change that thing. When the whole basis of the system needs to change, you can still keep all the components that don't need to change. The only future use case you should design for is change, because that's the only thing you can be certain of.

So don't make things more generic than you need today, but make sure it can be made more generic in the future. And in that future, don't just add new features all over the place, but reassess your design, and look if there's a more generic way to do the thing you're adding.

I do this sort of stuff all the time on my current project, and it works quite well. There's no part of the system we didn't rewrite at some point, but all of them were easy to rewrite.

Re: Ask HN: How to avoid over-engineering software design for future use cases?

#28
Data is the only thing that I personally care about "making future proof". Migrating data is a pain. Most everything else can easily be feature flagged or coded around.

Even then, I don't dig too deeply into things. I look for two things:

* What a migration path to realistically expected features _might_ look like. If I'm debating between two column types or table setups, go with the more flexible option.

* What scenarios will be extremely hard to get out of. Avoid those when reasonable.

----

In other words, instead of proactively planning for some feature to evolve into the unknown in some way. I'm looking to make sure I'm keeping flexibility and upgradability.

Re: Ask HN: How to avoid over-engineering software design for future use cases?

#29
Best practice is to follow the example of the subway system and build short spurs at the end of each line in the directions you might want to continue expanding. That way you're not incurring any real additional cost upfront, but you're saving a ton of money in the future if you decide to keep expanding.

Don't do work before you need it, but also don't give up optionality unless you're getting commensurate benefits from doing so.

Re: Ask HN: How to avoid over-engineering software design for future use cases?

#30
One heuristic I've adopted: when faced with a design decision, I ask the question "what's the simplest possible way to do this (that meets the requirements)?" So, do you need a fully event driven Kafka based system, or would a cron job running a python script be sufficient?

The followup is, if we have to change/replace said system, how difficult would that be? If scaling the simplest solution would be difficult/painful, then one can start look to higher complexity solutions. The idea isn't necessarily to always choose the simplest solution, but having it in mind can be helpful. It crystalizes one of the endpoints of the simple/complex spectrum and helps weigh the pros/cons of various approaches.

As a side note, it's sort of amusing that a lot of design focused interview questions are around things like "design a twitter/instagram like system that'll scale to billions of requests per day." I've never had to do anything like that IRL, but no interviewer has ever asked me to design, say, an invoicing system that gets called rarely. So perhaps one of the reasons the arc of software engineering bends towards complexity is that we're continuously rewarding a "build massive scalable systems" mindset?

Post reply on HN