Ask HN: How to avoid over-engineering software design for future use cases?

Designing for the future rarely works out very well. What does work, however, is encapsulation.

For example, in the 80's the linked list was my go-to data structure. It worked very well. But with the advent of cached memory, arrays were better. But my linked list abstractions were "leaky", meaning the fact that they were linked lists leaked into every use of them.

To upgrade to arrays meant changing a great deal of code. Now I encapsulate/abstract the interface to my data structures, so they can be redesigned without affecting the caller.

I was brought into to help at an early stage startup a few years ago. The company was building an e-commerce platform and the product owner had this idea of 'attributes' that could be attached to any kind of entity in the system (e.g. product, category, order, customer). If they needed a new attribute they would be able to simply set it up through the admin UI without any developer intervention (because developers are expensive!).

When I joined the attribute system had been build with a beautiful UI, and the backend was mostly working for managing attributes, but that was pretty much it. The first feature I was working on was showing products on the store, and for this the idea of attributes made sense. If you are selling a product in the "OLED TV" category you probably want a "Screen Size" attribute, and to be able to use it to compare against different products in that category. Through the platform we had maybe 500 product level attributes, with more being added all the time, so having them hard-coded wouldn't have been manageable. That was pretty much as well as it worked though.

Sellers needed to be able to manage their stock through the system, so on the warehouse entity there were attributes describing the number of products in stock, lead time, how often they restock, etc. The attributes didn't really have validations, but they had types which described what UI element should be displayed when they are entered. However all of the validations around that were at the whim of the front-end, and in some cases it would send what you would think should be a numerical type as a string (and then if you try to change it something would break as that expected it to be a string), so doing any kind of calculations or logic on the attributes was basically impossible. In the end I just added db-level fields to the stock entities, with validations in the backend to make sure these were as expected. The backend was a Rails app, so this took 10 minutes vs days trying to coerce the attribute system into doing what I needed.

As it was a Rails app we couldn't actually name the model of these Attribute, so had to give it another name, and whenever someone new joined (it was an early stage startup, so had high turnover) we had a 30 minute discussion explaining this. I never got the explanation of how the product owner expected logic to be attached to these attributes without a developer doing any work, but I'm sure they had an 'ingenious plan' for that too.

Needless to say, the startup burned through all it's funding without even launching, then managed to convince the investors to give them a little bit more, launched a half working product, and it turned out nobody wanted it.

> How far should we go in making things generic?

My rule of thumb for this is to repeat yourself at least 3 times before trying to build an abstraction to DRY them up.

3 use cases is obviously not always sufficient inform the design of a good abstraction, but IME:

- abstracting at 2 use cases is very often premature and results in leaky/brittle abstractions where the harm from added coupling between fundamentally incompatible usages far outweighs whatever little harm can be introduced by keeping the 2 usages as repeated code, and

- with any more than 3 use cases, the maintenance cost of keeping the usages in sync starts to scale non-linearly, so at the very least thinking about a design for a potential abstraction at that point becomes a worthwhile exercise, even if we end up deciding it's not yet appropriate (hence the "_at least_ 3 times").

But abstractions done right, can really help with refactoring.

But this is hard to do. I once made a major feature change - where I feared the consequences and bugtracking afterwards - but it worked like a charme, because the complex abstractions I made in the beginning, really helped, so it was done in a couple of days and not weeks or months, like I thought. I was really proud of my older me, that day.

Only, like you said, when you have layer of layer of abstractions, the problems rise. When you really have to dig in to understand what is going on. Or when the abstractions are simply not helpful for the new change, which happens way too often, too.

"Code is a liability. Be sparse. Build with some flexibility but overall just build what is required and when the future comes, change with it"

So yes, I very much agree to that.