Live data from Hacker News

Prefer duplication over the wrong abstraction (2016)

sandimetz.com

181–190 of 375 posts

Re: Prefer duplication over the wrong abstraction (2016)

#181
post #63
post #47

Earlier quoted context omitted.

Hardly iconoclastic, it's a very sensible suggestion. It would be iconoclastic if the common sense basic approach would be to start with abstraction. It's not, the common sense default is to write possibly duplicate behavior until you actually discover several cases to abstract away, until you bevalop a sensible idea of which functionality unites them and which doesn't carry over all of them. > Once you have an awkwa…

> Maintaining the wrong abstraction, or, god help, abstractions, would be even worse. Hard disagree. When you've had to chase through a change in untold and actually unknown numbers of duplications of code in different permutations and fix them because they are all on fire simultaneously, you'd disagree too. A bad abstraction would at least have had one fire in one place.

I, on the other hand, have had to burn through countless cycles of security alerts because I used a library for JSON parsing that had all kinds of other features that I didn't need or want.

The security bugs were all in features I never wanted.

A bit of simple duplication would have been golden.

Re: Prefer duplication over the wrong abstraction (2016)

#182
post #25

No it's not. This has always been a needlessly iconoclastic rather than sensible suggestion. At the very least it is not once you're working at the wrong kind of scale. Once you have an awkward number of customers (more than five and less than a hundred), maintaining duplicated code that should have been abstracted and modularised will only seem cheap if you don't mind that you burn through even junior employees at a…

One of the most challenging kinds of thought to work through with my engineers in professional communication is nuance. For example, they may say something like this, but actually mean "For a particular situation, this is wrong."

The context a decision is evaluated is particularly important for "rules of thumb" like this. There's the rule of 3 (which many senior engineers imparted to me earlier on in my career) - don't refactor until you've actually duplicated it thrice, but even so, what they speak of is a catch-22 that's pretty important to reason about carefully.

On one hand, if you overcorrected on the fear of abstraction, you could easily end up with 500 duplicates that are slightly different and need to be maintained 500 different ways, slowly causing slightly wrong behavior some of the time, data corruption, combinatoric explosion. Surely, once there is such a situation, some degree of abstraction is the only right decision.

On the other hand, if you overcorrected on the fear of duplication early on, you could easily end up with a premature optimization and complexity -- complexity which, most importantly, could be rooted in a gap of understanding of how the code will be used and what direction it may go in over time (often based on which direction the business will go over time).

The only answer that actually works, of course, is "somewhere in the middle." Obviously, that's pretty vague and not very useful. Where, exactly, in the middle IS the right place?

As the years have gone by, I've become more and more steadfast that the answer to that question is and must be an art and not a science. Of course, it must always be rooted in practicality, the actual context of the code around it and where the code/business was in the past and where it will be in the future.

But just as importantly, some of it must be based around beliefs in the face of imperfect information about what you want to invest in for the sake of the technology, the team that develops it, and the business that relies on it. It could be that for your team, your values make it make sense to go a little bit further than "good enough" on normalizing your data modeling, because the way you like to run your business requires that normal form to do the analytics and make decisions productively. It could be that for your team, your values make it make sense to go a little bit further than "good enough" on splitting service boundaries and ensuring clean queues and message passing infrastructure, because you have seasonal spikes where you need to scale up to a ton of load and then scale down after without constantly doing a song and dance or pre-provisioning fragile infrastructure.

But the most common thread there is - art, not a science. Every single decision depends on YOUR team, YOUR business, YOUR needs - and like any art, there is no universal rule or discovery or best practice in the industry that will magically work for your needs without working through the details of whether it appropriately fits your situation or not.

So with that said - I can't really agree with you. At any place I've ever worked with a competent team, maintaining duplicate code is just not that hard and follows the same process for being dealt with. Built a robust test suite that encodes the actual differences and the shared structure. Pull out the pieces that have a good reason to be abstracted and redesign the pieces that encode the true differential structure in a way that is intuitive. Lather rinse repeat. It's always straightforward because it's known - by the time you are doing this process, you've had tons of repetitions and data on what is driving you to develop the abstraction, so when you make the decision, you are making it empirically.

Conversely, I have seen many otherwise competent teams slowed to a halt with premature abstraction. Frameworks that were well intended and reduced duplication, but encoded coupling between components that at a certain point in the businesses progression, fought with reality rather than aided, and all because they were frozen into place before anyone empirically had really clear data about whether the abstraction would be worth it long term. Well intended "clean code" refactors that were meant to solve the old "bad duplication" but instead created a far more difficult to reason about "abstracted base" of code that didn't really solve any of the domain modeling problems and was just as difficult to maintain without introducing buggy behaviors (if not more so) than before.

The biggest problem is that premature abstraction is sexy and fun. There are incentives and dopamine hits from doing it extraneously. But fixing legacy duplication is not fun. And so when it gets done, it tends to get done in a pragmatic way to relieve pain rather than to elicit pleasure. That, I believe is one of the biggest confounding sociological aspects of this whole discussion.

Re: Prefer duplication over the wrong abstraction (2016)

#183
The bad thing with abstractions is when you start it too early in your code base for things. It's also a bad thing when you start it too late, although not as bad. If you start it way, way, way too late it's very, very, very bad.

Of course, the worst abstractions are the ones you don't need at all.

Re: Prefer duplication over the wrong abstraction (2016)

#184
post #74

Earlier quoted context omitted.

> I would go as far as to say that any abstraction you can maintain (that is in active maintenance, I mean) is better than code duplication once you are passed a de minimis threshold. Pretty much everyone arguing for duplication has argued what you are saying, which is wait to see a few instances of it before committing to an abstraction. No one is saying duplicate everything 100 times. So I don't think this discussi…

The point is it sounds all smart and sophisticated and principled in the abstract environment of a code discussion in a blog post. In the real world, duplication happens in an emergent way, there isn't the time each time to judge whether it's really time to just quietly abstract that code, you may not get the permission, budget or window to do it, and if you don't stop the rot really early you are locked into the pat…

But... it shouldn't. People are arguing that a bad abstraction is better than none at all. Badly-implemented abstraction is the same. If you hit code that is duplicated organically a dozen times, you don't make it a baker's dozen. You spend a bit of extra time at least stubbing out the abstraction so future organic duplication can at least share an entry point. Abstractions grow organically too, in well-tended codebases.

Re: Prefer duplication over the wrong abstraction (2016)

#185
post #63
post #47

Earlier quoted context omitted.

Hardly iconoclastic, it's a very sensible suggestion. It would be iconoclastic if the common sense basic approach would be to start with abstraction. It's not, the common sense default is to write possibly duplicate behavior until you actually discover several cases to abstract away, until you bevalop a sensible idea of which functionality unites them and which doesn't carry over all of them. > Once you have an awkwa…

> Maintaining the wrong abstraction, or, god help, abstractions, would be even worse. Hard disagree. When you've had to chase through a change in untold and actually unknown numbers of duplications of code in different permutations and fix them because they are all on fire simultaneously, you'd disagree too. A bad abstraction would at least have had one fire in one place.

A bad abstraction would have caused many updates in many places because the API would never quite stabilize due to having been a force-fit from the start.

A uses the abstraction, but finds the API doesn't work. Fixes that.

That causes B to have to make a tracking change which induces a bug. B realizes that the API isn't quite right. Fixes it.

That causes A and C to make tracking changes. These induce more bugs. C fixes the abstraction to avoid these cases.

This breaks A and B so they decline to update.

And so on. This is what a bad abstraction looks like. API "fixes" bouncing around the code as they reflect off of the bad abstraction.

Re: Prefer duplication over the wrong abstraction (2016)

#186
post #2

You can do both with microservices!

I get the joke, but in ideal world, in microservices, there is no such thing as code duplication across services. As a maintainer of a service, I should not give a crap about code present in some other service - it's some other team's code, why would I care? I don't have to even know that the other team exists. In big systems, it happens that I can't even feasibly know the existence of all the applications.

Re: Prefer duplication over the wrong abstraction (2016)

#187
post #2

You can do both with microservices!

Except 9/10 times microservices end up wildly dependent on each other, yielding a distributed monolith. Better to use service oriented architecture and just ship the monolith, you can test easier and skip the extra layers of serialization / deserialization.

> end up

So it just happens, right? There is no remedy to this? You know the answer :)

BTW I'm all for monolith.

Re: Prefer duplication over the wrong abstraction (2016)

#188
post #25

No it's not. This has always been a needlessly iconoclastic rather than sensible suggestion. At the very least it is not once you're working at the wrong kind of scale. Once you have an awkward number of customers (more than five and less than a hundred), maintaining duplicated code that should have been abstracted and modularised will only seem cheap if you don't mind that you burn through even junior employees at a…

I'd recommend clicking through the headline to watch the talk. Metz talks a lot about types of similarity: similarity by coincidence vs similarity due to an actual semantic or functional equivalence. Code that is coincidentally similar very often diverges in either the short or long term, and DRYing it up aggressively tends to result in functions that have many boolean parameters that each trigger disjoint sets of be…

> I'd recommend clicking through the headline to watch the talk. Metz talks a lot about types of similarity: similarity by coincidence vs similarity due to an actual semantic or functional equivalence.

I've seen this article and AFAIR the video before, and FWIW having been a Rails developer from the very early days and fitfully until maybe even 2014, I now interpret the phrase "my Railsconf talk…" quite negatively.

ETA: nice to be back to disagreeing with people on HN about coding principles again though. Hopefully this is a sign.

Re: Prefer duplication over the wrong abstraction (2016)

#189
post #55

I believe that "single source of truth" is a principle that should always be followed. If there's duplicated code where it'd be a bug if they diverge, then you should refactor. It creates a long-distance coupling in your code that may be invisible to future developers until a bug emerges. But with that in mind, I mostly agree with the article: if it's not a violation of "single source of truth", then abstractions are…

With LLMs the cost of duplication is much lower and LLMs

Re: Prefer duplication over the wrong abstraction (2016)

#190
post #2

You can do both with microservices!

I get the joke, but in ideal world, in microservices, there is no such thing as code duplication across services. As a maintainer of a service, I should not give a crap about code present in some other service - it's some other team's code, why would I care? I don't have to even know that the other team exists. In big systems, it happens that I can't even feasibly know the existence of all the applications.

You mean you don’t have 300 versions of your badly developed rapidly evolving platform services rotting away underneath the bit you didn’t duplicate?
Post reply on HN