Garbage collect your technical debt (2021)

Consider the alternative perspective that each new feature actually makes it a new piece of software. You were able to fully design and specify the original piece up front, thus avoiding technical debt on that piece of software.

It seemed like the word iterative became a euphemism for “dont worry about it”. I have worked across the entire industry for 30 years, and I have never seen a “true iteration”. On any feature or application. Anything at the end of the sprint that doesnt work is just a bug. Thats not iteration.

Iterative development was just a powerpoint slide for the Scrum Industrial Complex which was hard selling itself and it worked!

This article which seems even more antiquated than Id like to describe, seems to confuse iterative with “having all the requirements at once”. Iteration and the quality of requirements, and the bandwidth to analyze them and design a system dont really have much to do with each other.

And thats why iterative came to be a stand in for “dont worry about it”.

I personally feel that the creators of Scrum or some responsible group of modern architects, should audit the results of supposed iterative development, and do an honest assessment of what it actually is in practice. Because its not what we were sold, and its not iterative. It does result in awfully bad software but there have been other shifts in development practice that have accelerated the awfulness of delivered work in the industry, and Ill just leave it at that.

Please resist the temptation to comment, “because your not doing scrum right”. That was the genius of the Scrum Industrial Complex, to suggest that somehow this simple thing is really magical and complex and keep repeating the same things over and over. Its not magical or complex, its just bad.

As far as the article goes, I have been on projects that did spend over a year ingesting requirements and doing design before starting to code. Its true they usually fail. But thats a money/commitment/time to market problem.

Starting to “code” right away creates some better optics and fewer arguments in meetings. The one who pays the piper calls the tune as always.

"and write tickets for it, so it can be seen and brought up for discussion, prioritized against other tasks"

I have been doing this for almost a decade now and I have at no point ever seen even a top-ticket tech debt item get prioritized. Ever. It's always too much effort for little payoff (in the eyes of the business).

I explicitly tell juniors that when I cut this kind of ticket, I do it because it is right, not because I think it will ever get worked unless I myself (or someone similarly insane) directly disobey marching orders to work on it.

And even when someone does disobey orders and refactor, it's often a pet peeve that isn't one of the more major ones. So that's fine but nobody ever one-man-armies against the real problems.

And to illustrate what I mean by "the real problems", let me explain. I like to talk about tech debt as high interest or low interest.

Low interest tech debt is the shitty python script I wrote the other day that took 1-2 hours of manual monkey work we'd have to do in Presto sql and that only 1-2 people even knew how to do, and makes it so literally anyone at the company can do it perfectly right in a minute or two. It's a shitty script, but it knows it's shitty, and tries to have no pretenses about it and make it as simple as possible to debug and make incremental changes to it. So yes, if I spent more than an hour or two on it, I would not have done it like this at all. And yea it'll probably be a little annoying to change the next time someone has to do it. But it saves a lot of time, didn't cost much, and it's not going to change how we design the system at all.

That last bit hints at what I mean by "high internet tech debt". High interest tech debt is how it takes 8 hours of babysitting to deploy a schema change. It causes blip outages in every region as you deploy it. It's so painful it perverts our schema design to minimize how often we have to migrate the database. And it's all because, shortly after the Big Bang, right in between photons existing and the first hydrogen nucleus existing, someone had a cool idea for a simple trick about how they could eat their cake and have it too with how our databases work. And we have paid for it ever since. And they built on top of the primitives this offers so deeply that we'd have to change a ton of shit to make it work. The best time to fix this was 5 years ago, the second best time is now, and I guarantee you we will never ever do it. We'll replace the whole thing with something else because it's shiny and will be worse for years before we actually just pay down this tech debt