Live data from Hacker News

Prefer duplication over the wrong abstraction

sandimetz.com

51–60 of 101 posts

Re: Prefer duplication over the wrong abstraction

#51

Earlier quoted context omitted.

Agreed. I think this is actually dangerous advice. While I agree that the wrong abstraction can cause lots of pain, the repetition of a bug through duplicate code is arguably a worse issue. One example might be not using an abstraction for accessing filesystem resources in a web based service. You might end up duplicating a ton of code that improperly takes a string from a request without sanitizing it and then have…

This would not be a candidate for duplication tho, because the underlying intent is also logically the same. Author is talking about things that coincidentally happen to have similar code at the moment, but are fundamentally and logically unrelated, having a high likelihood of diverging in the near future.

I understand the intent. I still believe the advice is dangerous if improperly applied.

Re: Prefer duplication over the wrong abstraction

#52
post #40

This made me throw up in my mouth a little. Duplication is not inherently wrong as there is a point where you should stop abstracting and live with repetitive code, but that point tends to be way further out than most inexperienced devs would assume. If you're using duplication as a temporary tool to help find the right abstraction, fine. But don't check that code into source control. The problem with duplication is…

The problem with premature abstraction is that it becomes difficult or impossible to reabstract since you're not looking at the root issue anymore but rather attempting to unroll a failed abstraction.

Often by the time the abstraction is such a failure that it needs to be rewritten, it's evolved into its own system with a menagerie of hacks latched on to let it live another day. You're not going to just come up with a regeneralization because more often than not it's not generalizing anything anymore.

Seems a bit much to "throw up in your mouth a little". Premature abstraction and the high cost of indirection are responsible for some of the hardest problems in software and you don't offer a solution beyond "try anyways".

Re: Prefer duplication over the wrong abstraction

#53
This also applies to CSS and is the reason why most CSS codebases are a mess. Functional CSS mitigates this (at the cost of duplication, but personally I've found it much more easy to maintain).

The original problem comes from trying to map logical components to CSS classes and forcing inheritance. This is a good intention but doesn't work well. With the project evolving, it gets complicated to alter something without breaking something else, because the inheritance chain means that rule conflicts are likely and not that easy to spot, so to have a mental model of how the element will behave visually means a lot of cognitive load.

Re: Prefer duplication over the wrong abstraction

#54
post #45
post #21

I see the author's point, but it overlooks code maintenance. At step 4, where "Time passes" it should read, Programmer A fixes bugs, optimizes, adds features, etc. Repeat. If the code is duplicated, the maintenance effort is also duplicated or the duplicates diverge significantly making them more difficult to understand in relation to each other. Why does methodA do this, but methodB does that? Tools can help prevent…

Programmer A replaced the duplication with a new abstraction at step 3, so maintenance activities can be performed once on the abstraction definition instead of once per copy. It wasn't until the functionality diverged (thereby invalidating the abstraction) that the author advises replacing the abstraction.

>Programmer A replaced the duplication with a new abstraction at step 3

Why would deduplication stop maintenance? Unexpected NullPointerExceptions are found. Unexpected UTF-8 string inputs are discovered. An interger overflow is discovered. A memory leak is found in the method. The method is used in a performance critical loop and now must be optimized.

Maintaining code is the majority of programming work that I've witnessed. If multiple copy/paste methods have to be maintained, it always devolves into a mess of technical debt until someone comes along, pays the debt, and pulls them all up into the same abstraction.

Re: Prefer duplication over the wrong abstraction

#56

I'm so glad to hear a pithy saying given to this. So many times I've fought for ripping out some indirection-heavy pattern and just doing {thing} procedurally, only to run up against the concern of the duplication it'll introduce.

This doesn't validate the procedural position in a procedural vs polymorphism debate; if anything, the example can be summarized by "there was a good polymorphic abstraction, programmer B came along and couldn't figure out how to extend the abstraction to implement his new requirement, so he made the abstraction less polymorphic and more procedural, and then this happened over and over again until the accumulated procedural logic made the whole thing unbearably bad".

I think the appropriate summary for this article would be: bastardized abstraction < duplicated logic < proper abstraction.

Re: Prefer duplication over the wrong abstraction

#57

Man I dunno. This is so tough. On one hand, absolutely. Stacking params on params is a recipe for disaster. But often times with duplication you do need to update the duplicates. Which means you wind up with 6 pieces of code all similar but slightly different such that it fails in increasingly nuanced and edge casey ways. No silver bullet unfortunately.

You don't have to choose. If you have a function* with excessive arguments, it has too many responsibilities. Identify the responsibilities and make a function for each. Then replace the call sites of the original function with only the new functions that are appropriate for that use case. If you find that the same N new functions are repeatedly called in the same places, you can pull those out into a new function as well. It's all very simple, though learning to decompose responsibilities correctly takes practice.

* For sanity's sake, I'm just going to say "function" in place of "abstraction".

Re: Prefer duplication over the wrong abstraction

#58
post #54
post #45

Earlier quoted context omitted.

Programmer A replaced the duplication with a new abstraction at step 3, so maintenance activities can be performed once on the abstraction definition instead of once per copy. It wasn't until the functionality diverged (thereby invalidating the abstraction) that the author advises replacing the abstraction.

>Programmer A replaced the duplication with a new abstraction at step 3 Why would deduplication stop maintenance? Unexpected NullPointerExceptions are found. Unexpected UTF-8 string inputs are discovered. An interger overflow is discovered. A memory leak is found in the method. The method is used in a performance critical loop and now must be optimized. Maintaining code is the majority of programming work that I've w…

> Why would deduplication stop maintenance?

I didn't say it would, I said it would stop repetitive maintenance--meaning you won't have to update many duplicates of the same logic because there aren't duplicates.

I tried to update my comment to be more clear, but I'm not sure that I succeeded. :p

Re: Prefer duplication over the wrong abstraction

#59
post #30

For a long time I've thought that Don't Repeat Yourself is the most important software engineering rule. I still do, but in line with this article, I've learned that some things that appear to be duplication if you just look at the literal code actually aren't. Just because you've got a chunk of ten or twenty lines that are identical right now doesn't mean that they are actually identical. It's hard to give concrete…

I agree. I follow the DRY principle very strongly, but not on the literal code level - just one level above, on the "what is this doing" level. So in your example, if I see a dozen places where the program is using that URL library to e.g. send requests to get a different dataset from the same API, I'll quickly DRY it into a function. That's because all those places do the same thing - "fetch a dataset named $name from API XYZ". But I'll leave alone all the other places that use the same URL library, even with pretty much identical parameters, if they're doing different things. They may look the same now, but they may start to differ later. And even if they don't, placing them under a common abstraction when they represent different things is basically a lie. I don't like it when the code lies to people.

Re: Prefer duplication over the wrong abstraction

#60
post #30

For a long time I've thought that Don't Repeat Yourself is the most important software engineering rule. I still do, but in line with this article, I've learned that some things that appear to be duplication if you just look at the literal code actually aren't. Just because you've got a chunk of ten or twenty lines that are identical right now doesn't mean that they are actually identical. It's hard to give concrete…

There's a reason why Torvalds said, "Talk is cheap. Show me the code."[1] Quite often when people write about abstract concepts in software that violate well-known principles, they cannot come up with concrete examples that show a good time to break form. Because they are few and far between.

Take URLs for an example.

> Here I may be deeply concerned about SSL

On web pages, instead of codifying "http://" and "https://" everywhere (a form of duplication), it is possible to drop the protocol to use "//"; the browser will fetch subsequent files using the security of the original page. (Mixing secure and insecure content is an architectural mistake.)

An application shouldn't be concerned with HTTP vs HTTPS. In Java, that concern is made transparent by HttpURLConnection (and its subclass of HttpsURLConnection), for example.

> a chunk of ten or twenty lines that are identical

When seemingly identical chunks of code are repeated, they can often be abstracted and parameterized. Lambda expressions help reduce duplicated code snippets. Of course, when I say "parameterized" I don't mean "pass in a metric ton of parameters" as there are techniques for shortening long parameter lists (such as multiple methods and the builder pattern).

Lines 70 to 89 are an example of duplicated code[3] (here are four lines, but you get the idea):

    this.namespacesAware = xmlNode.getAttribute("namespacesaware");
    this.prunetags = xmlNode.getAttribute("prunetags");
    this.hyphenReplacement = xmlNode.getAttribute("hyphenreplacement");
    this.booleanAtts = xmlNode.getAttribute("booleanatts");
These should be a map of properties rather than a volley of String instance variables:

    private String namespacesAware;
    private String prunetags;
    private String hyphenReplacement;
    private String booleanAtts;
It might not seem like much duplication at first, until you look at the code that uses those parameters[4]:

        final String namespacesAware = BaseTemplater.evaluateToString(elementDef.getNamespacesAware(), null, context);
        if (namespacesAware != null) {
            properties.setNamespacesAware(CommonUtil.isBooleanTrue(namespacesAware));
        } else {
            properties.setNamespacesAware(false);
        }

        final String hyphenReplacement = BaseTemplater.evaluateToString(elementDef.getHyphenReplacement(), null, context);
        if (hyphenReplacement != null) {
            properties.setHyphenReplacementInComment(hyphenReplacement);
        }

        final String pruneTags = BaseTemplater.evaluateToString(elementDef.getPrunetags(), null, context);
        if (pruneTags != null) {
            properties.setPruneTags(pruneTags);
        }

        final String booleanAtts = BaseTemplater.evaluateToString(elementDef.getBooleanAtts(), null, context);
        if (booleanAtts != null) {
            properties.setBooleanAttributeValues(booleanAtts);
        }
The entire code block could be reduced to one line of code had properties been used instead of instance variables. (Maybe a loop and two lines of code, but either way the duplication could be easily mitigated.)

Without seeing source code examples, it is extraordinarily difficult to identify problems in the code and explain how the code could be improved. Hence the Torvalds quote.

> the same abstraction is just pain

Sometimes code is poorly designed, not designed for extensibility, or has hard-coded assumptions that should be exposed as properties. Take line 52 of the HtmlToPlainText class[2]:

    private static final int maxWidth = 80;
That one line forced duplicating the entire class, which was otherwise perfect for my needs. But the class did not expose behaviour to change the maximum width of a line. An incorrect assumption was made at the design level: 80 characters ought to be enough.

The correct solution would be to make it an attribute and send a pull request to the author. :-)

[1]: https://lkml.org/lkml/2000/8/25/132

[2]: http://grepcode.com/file/repo1.maven.org/maven2/org.jsoup/js...

[3]: https://sourceforge.net/p/web-harvest/code/HEAD/tree/trunk/w...

[4]: https://sourceforge.net/p/web-harvest/code/HEAD/tree/trunk/w...

Post reply on HN