A Field Study on Technical Debt • Hacker News Board

Re: A Field Study on Technical Debt

#71

Post by keithpeter » Wed, Jul 29, 2015, 3:18 PM UTC

ryandrake wrote: post #67

keithpeter wrote: post #52

Earlier quoted context omitted.

Sounds like a small company making a niche product. What were you able to do to improve the situation? Which issue did you tackle first (documentation of processes backwards or source control forwards)?

I'd like to think I left the situation far better than I found it. Tackled source control and issue tracking first since they were low hanging fruit. Next came basic (manual) backups. My background is not in putting together general office-wide IT solutions, so I did not feel qualified to recommend any specific package. After that, it was on to a sane build and deployment process that involved a dedicated build machi…

Thanks for reply. Software for manufactured systems is an area that can only grow so I hope more get involved in the way you did.

Re: A Field Study on Technical Debt

#72

Post by wellpast » Wed, Jul 29, 2015, 4:13 PM UTC

dasil003 wrote: post #70

wellpast wrote: post #69

Earlier quoted context omitted.

This is the fallacy I was referring to--architectural soundness has nothing to do with anticipating change. The "choice" your engineer faces to shoehorn a feature in [1] versus implement it in a way that optimizes for soundness (decoupled code w/ narrow interfaces) is not a choice if the skill set is there--i.e., if he/she has trained and practiced the right skill set. [1] (By 'shoehorn' I assume you mean implement i…

You can't just reaffirm that without evidence and an argument. It is self-evident to me (and I presume every other software engineer here) that different architectures lend themselves to different types of change. If you want to make that claim than you probably should write a dissertation about it because it's quite a controversial claim. But simply stated as if it were unassailable fact is pushing you right into cr…

> It is self-evident to me that different architectures lend themselves to different types of change.

This is such a vague statement, I have no idea what you mean or how it relates to our discussion.

My argument is this:

(1) The most costly issue with a code base has to do with its architectural properties. How architecturally sound it is [1]. Most other issues that we tend to categorize as technical debt (like code duplication or verbose code, for examples) pose such tiny cost in comparison to the cost of poorly architected code [2].

(2) Architectural soundness is an objective property of code independent of the code's generality, flexibility, etc. Again, it's the degree to which its components and its interfaces narrow.

(3) There exists a skill set that understands, recognizes and produces architecturally sound code. [3]

(4) Given an architecturally sound code base and a set of features, a skilled person can implement the features without violating the architectural soundness of the code base AND do so without having to pay for it in time-to-delivery. [4]

I think I'm being fairly clear in my assertions here. They are based on my long experience. To disagree is to pose a different theory. Do you care to elaborate on what you mean by 'crackpot territory'?

---

[1] More specifically, the degree to which its components (modules, classes, functions) are coupled and their interfaces not as narrow as they could be. This is an objective property of a code that has clearly delineated components (i.e., you can quantify dependencies, interface sizes, module sizes.)

[2] Yes, this is a claim based on my 20 years of experience working in myriad code bases. How to distill and prove this, formalize the argument, etc is probably work to be done.

[3] This skill set doesn't come cheap and requires proper training, and currently our industry/academies aren't producing it.

[4] Again, this claim based on my experience and yes there is probably some good research opportunity here to formalize it.

Re: A Field Study on Technical Debt

#73

Post by dasil003 » Wed, Jul 29, 2015, 7:52 PM UTC

wellpast wrote: post #72

dasil003 wrote: post #70

Earlier quoted context omitted.

You can't just reaffirm that without evidence and an argument. It is self-evident to me (and I presume every other software engineer here) that different architectures lend themselves to different types of change. If you want to make that claim than you probably should write a dissertation about it because it's quite a controversial claim. But simply stated as if it were unassailable fact is pushing you right into cr…

> It is self-evident to me that different architectures lend themselves to different types of change. This is such a vague statement, I have no idea what you mean or how it relates to our discussion. My argument is this: (1) The most costly issue with a code base has to do with its architectural properties. How architecturally sound it is [1]. Most other issues that we tend to categorize as technical debt (like code…

> Architectural soundness is an objective property of code independent of the code's generality, flexibility, etc. Again, it's the degree to which its components and its interfaces narrow.

I like this definition, and I agree you could quantify this into a useful measure. However the optimal architecture according to this measure is a function of the problem domain. You split the modules and define the interfaces to maximize the encapsulation and minimize the interfaces.

So what happens when the business rules introduces a new cross-cutting concept? The basis of the architecture soundness is now in question. Do you define a new module and add n new interfaces? Or do you split the concern into the existing modules, fattening them in the process, and widening existing interfaces as well?

Re: A Field Study on Technical Debt

#74

Post by nernst » Wed, Jul 29, 2015, 8:35 PM UTC

PopeOfNope wrote: post #25

stevecalifornia wrote: post #3

I have never worked in an environment where new employees didn't show up and declare that the existing infrastructure sucks. Knowing this, I caught myself doing the same thing. Reflecting on it, for me it was a coping mechanism. I was overwhelmed by all the new things and didn't understand why this was setup that way, etc. The reason I bring this up is I am surprised that architectural choice is overwhelmingly the to…

Notice that nowhere in the article is there any mention of what a good architectural choice is , nor any mention of the perception of debt in relation to the size of the system[0]. My personal experience with architectural choices is that all of them are great early on when the code base is small. As that code base grows, they all suffer under the weight. [0]: 100k-1M is a very large range. Kudos to them for filterin…

(author here) We cross-tabbed system age vs perception of amount of TD. There was a moderate association between older systems (> 6 yrs) and more perceived debt.

I did not explicitly look at size as this was not one of the original research questions, but a good point. I suspect older systems will tend to be larger (in the domains we studied, anyway). And your point about arch choices being great "early on" can, I think, be captured in the "system age" variable. I guess I'm trying to think of a system that might be young and yet quite large in LOC.. would be an interesting outlier to look at.

Re: A Field Study on Technical Debt

#75

Post by nernst » Wed, Jul 29, 2015, 8:39 PM UTC

bglusman wrote: post #2

Shameless plug: I didn't know about their SonarQube project until I'd built debt ceiling, but while ruby only at the moment, it is an open source attempt to provide some mechanics for defining and tracking some quantified approximation of technical debt. No claims it's perfect of course, but would love more real world feedback and/or feature requests! Also would like to extend it to handle JavaScript analysis as well…

I like the tool. As for the extensions, I would suggest that in my experience SonarQube has the same ability for extension, and does most of its work in the open. So it might be worth considering turning your project into a plugin (or plugins) for SonarQube?

Re: A Field Study on Technical Debt

#76

Post by Philadelphia » Wed, Jul 29, 2015, 10:20 PM UTC

SideburnsOfDoom wrote: post #54

ryandrake wrote: post #39

Earlier quoted context omitted.

I took a development job once, where I learned on the first day: 1. No backups. Emergency plan = several older copies of the product's source spread among hard drives of various former employees' systems. 2. No source/revision control. Current version of the product's source code was literally whatever was on the lead developer's hard drive at the moment. 3. No deployment process. When a release had to happen, above…

>took a development job once, where I learned on the first day: No backups, No source/revision control, No deployment process, No dependency management, No bug tracker. And that is why one should always ask about these important things before the first day, i.e. during the interviews, before you decide if you want the job or not. Fortunately I'm in a position that the above would be a near-automatic "no" from me. edi…

Unfortunately, often the answers you get when you ask in an interview bear very little relation to the reality you find when you start a job.

Re: A Field Study on Technical Debt

#77

Post by mikekchar » Thu, Jul 30, 2015, 12:35 AM UTC

PopeOfNope wrote: post #47

mikekchar wrote: post #38

Earlier quoted context omitted.

The thing that is very frustrating in the "There was never time to change decisions..." bit is that there often is time. If you are curious about what I mean, I invite you to try a little experiment. For a few weeks (2-4 should be sufficient) tell people, "If you see something wrong with the design, refactor it as soon as you see it. If you need some extra time on your story to compensate for that work, just bring it…

For the people who don't refactor, the reason they usually give is: "There is no time to refactor". My #1 reason to avoid refactoring code is because that code has been there, is battle tested and hasn't had a bug filed against it in months. That is not my first instinct, either. There are lots of things I see that I had written months ago that I badly want to rewrite every time I see it. I have to restrain myself be…

Certainly it is always a judgement call. I was just saying to my colleague yesterday that where a developer really makes a difference on a project is by consistently being able to make the right judgement calls. One can say there is no silver bullet, and while it is impossible to make a project go faster than it can, it's very possible to make it go orders of magnitude slower ;-)

On that note, a few things I try to keep in mind: As you say, don't gratuitously change code. You may hate the design, but if it ain't broke, don't fix it. This is probably the biggest mistake that "change the world" programmers make.

Second, try not to rewrite code -- ever. Usually there is no business case (see "don't gratuitously change code"). Even if you think there is a business case, it dramatically increases risk. My rule of thumb: anything that lasts longer than 2 weeks has a very likely chance of being cancelled. If you must rewrite, it has to take much less time than that.

Finally, keep in mind that refactoring is not rewriting, nor redesigning (though it is closer to the latter than the former). Refactoring is transforming the code so that it performs exactly the same function (bugs and all!!!) with a different "shape". Ideally you will have tools to help you refactor in such a way so that you can prove that the resultant code executes in exactly the same way as the original code. With or without the tools, you should have a suite of unit/integration tests that will alert you when you have made a mistake.

Refactoring allows you to slowly migrate code from one "shape" to another over time while not breaking it. People who are skilled at refactoring can evolve efficient design even starting with really badly written code.

Why do you want to do refactoring? While, as you point out, the cost of doing a work-around from a sub-optimal design is low, the cumulative cost of these work-arounds over time can be quite substantial. A work-around introduces complexity to the code. This complexity makes everything slightly more difficult and slightly more risky. It also makes further work-arounds more likely. These work-arounds compound the problem. Because poorly designed code usually has high coupling, problems in one area of the code can manifest in other areas without warning. As the work-arounds increase, the complexity can increase exponentially. As an example of a worst case scenario, I once worked on a project where the programmers averaged 1 line of code per day (Yes, LOC is a poor measure of productivity, but no matter how you slice it, that's just incredibly bad).

My experience has been that teams which refactor effectively outperform similarly skilled teams who don't refactor by a very large margin. Although productivity metrics are impossible, the difference is quite startling. Interestingly, I have also had some experience with teams that have very high test coverage, but which don't refactor consistently (or effectively). These teams also do not seem to benefit from dramatically improved performance. My current theory is to write tests to support refactoring and don't worry about any of the other benefits.

YMMV :-)

Re: A Field Study on Technical Debt

#78

Post by wellpast » Thu, Jul 30, 2015, 1:07 AM UTC

dasil003 wrote: post #73

wellpast wrote: post #72

Earlier quoted context omitted.

> It is self-evident to me that different architectures lend themselves to different types of change. This is such a vague statement, I have no idea what you mean or how it relates to our discussion. My argument is this: (1) The most costly issue with a code base has to do with its architectural properties. How architecturally sound it is [1]. Most other issues that we tend to categorize as technical debt (like code…

> Architectural soundness is an objective property of code independent of the code's generality, flexibility, etc. Again, it's the degree to which its components and its interfaces narrow. I like this definition, and I agree you could quantify this into a useful measure. However the optimal architecture according to this measure is a function of the problem domain. You split the modules and define the interfaces to m…

> However the optimal architecture according to this measure is a function of the problem domain.

I agree with this. But domains typically evolve incrementally -- and using the skillsets I mention you can evolve your code accordingly. [1]

There is then the question of these cross-cutting, earth-shattering domain changes you mention. I'm not convinced my argument doesn't have a play here but regardless if these kinds of major domain changes are happening frequently enough to matter in this discussion then I think there's a different business-level problem going on.

It's hard to talk abstractly about this for so long--until of course someone gets to building out these ideas more formally. Getting into concrete cases would be more fruitful.

---

[1] That you can without cost tradeoff is my argument. Today these skills are so rare so you don't find orgs/codebases that do this properly--and therefore we have this fallacy that because it's prevalent that it's somehow necessary. We don't realize it's really a skills problem.

Re: A Field Study on Technical Debt

#79

Post by PopeOfNope » Thu, Jul 30, 2015, 10:26 AM UTC

nernst wrote: post #74

PopeOfNope wrote: post #25

Earlier quoted context omitted.

Notice that nowhere in the article is there any mention of what a good architectural choice is , nor any mention of the perception of debt in relation to the size of the system[0]. My personal experience with architectural choices is that all of them are great early on when the code base is small. As that code base grows, they all suffer under the weight. [0]: 100k-1M is a very large range. Kudos to them for filterin…

(author here) We cross-tabbed system age vs perception of amount of TD. There was a moderate association between older systems (> 6 yrs) and more perceived debt. I did not explicitly look at size as this was not one of the original research questions, but a good point. I suspect older systems will tend to be larger (in the domains we studied, anyway). And your point about arch choices being great "early on" can, I th…

I guess I'm trying to think of a system that might be young and yet quite large in LOC.. would be an interesting outlier to look at.

Take a look at video game companies. They tend to build up a large infrastructure early in the development of a game.

Keep up the great work! I love to see scientific research being done on software development. :)

Re: A Field Study on Technical Debt

#80

Post by PopeOfNope » Thu, Jul 30, 2015, 11:26 AM UTC

mikekchar wrote: post #77

PopeOfNope wrote: post #47

Earlier quoted context omitted.

For the people who don't refactor, the reason they usually give is: "There is no time to refactor". My #1 reason to avoid refactoring code is because that code has been there, is battle tested and hasn't had a bug filed against it in months. That is not my first instinct, either. There are lots of things I see that I had written months ago that I badly want to rewrite every time I see it. I have to restrain myself be…

Certainly it is always a judgement call. I was just saying to my colleague yesterday that where a developer really makes a difference on a project is by consistently being able to make the right judgement calls. One can say there is no silver bullet, and while it is impossible to make a project go faster than it can, it's very possible to make it go orders of magnitude slower ;-) On that note, a few things I try to k…

We're saying the same things, just in different ways. I was attempting to be humorous in my previous comment, preferring comedy to precision. :)

Interestingly, I have also had some experience with teams that have very high test coverage, but which don't refactor consistently (or effectively). These teams also do not seem to benefit from dramatically improved performance. My current theory is to write tests to support refactoring and don't worry about any of the other benefits.

That's something I had never considered. Do you have any general guidelines for testing for refactoring?