Garbage collect your technical debt (2021)

I'd argue that there's definitely a business case for frequent and early refactoring - it just needs to be refactoring worth doing in the first place. IMHE a most code gets touched rarely, and some parts get touched all the time, so refactoring needs to be strategic if it's going to have any utility.

This also argues for ongoing refactoring - addressing pain points as they arise, while they are fresh in people's minds, rather than suffering through them until Stockholm Syndrome sets in, people can no longer see the forest for the trees, and much of the velocity refactoring would provide is lost because the subsequent work is already done.

But the biggest issue I've seen with not handling tech debt on an ongoing basis is that there's never a good time to start. So if it's not built into the development cadence, then resistance builds for doing it at all. Product starts pointing fingers at "slow" engineering, who point back at the "breakneck" demand for feature work, and negotiations start for unrealistic (for both sides) halts in feature work - neither sufficient to resolve the problems nor short enough to avoid hurting the business.

Then product breathes a sigh of relief - the tech debt is "resolved" and will never need to be addressed again, engineering returns to wading through a codebase that is only marginally less swampy than it was before the cursory refactoring sprint, and the downward spiral (and finger pointing) resumes.

At least that's how it always seems to happen around me :)

As far as tests, IMHO refactoring without them (with BDD style tests being greatly preferable) is fraught with peril. But unless the team is bought in on BDD tests and using them to guide development, I agree they are a time sink to write them early. However, writing them later (and around code that might have been touched by multiple hands) rather than maintaining them is flavor of pain - like the refactoring, it's harder to be sure they won't miss things and you'll break something.

what was your response?

I'd definitely agree with the putting the refactor into a separate PR and then the new functionality in another.

Aside from the obvious that people are more likely to be willing to review your change when it's small, it makes it much easier reason about both when they are separate.

There will be a lot of noise generated from the refactor that will drown out the new feature, but self-contained in its own PR it's a lot easier to understand the new feature and spot mistakes.

A simple refactor can likewise be likely skimmed over quickly, looking for the patterns of how it was done and assuming that most of the changes were similar and mostly just looking out for the differences.

As for accepting the feature PR but not the refactor PR, that suggests that the feature doesn't actually rely on the refactor. In that case, it's even clearer that they should be separated.

Personally, I'd always even create a new ticket for the refactor so that there's some justification for the work, and maybe you can say that this ticket blocks the one for the actual work. Maybe that's just me though, because I like to make sure that every non-trivial commit is tied to an issue in the bug/issue tracking software. This means that every bit of code is a simple "git blame" away from a justification of why it was changed.

> By all means, have such a check as a compile-time error, but not as something that gets shipped to customers.

Tests don't get shipped to customers.

In my experience you need to do ‘the big refactor’ when a codebase has to handle something big that was not part of the original design such that you find yourself having to break the simplicity, elegance, completeness, coherence, etc of the existing system by tacking on some lopsided or alternate route or structure. What you really want in place of doing the ‘tack-on’ is a new simple, elegant, coherent, etc system that can handle both the old requirements and the new. In other words, you want to do the big refactor when you have new requirements that really should have been known at the time the system was designed such that you would have done things differently to accommodate them along with all of the requirements that _were_known at the time. This is easier to do the more monolithic and strongly-typed the application is.

Naturally then, you do want to know as many requirements up front as possible, which is the basic point of the article. Even though it’s not always possible, it’s still the best path to try.

All of this ‘screw design lets just roll up our sleeves and start coding’ is a great way to end up with spaghetti code and technical debt.

The key is to do as much requirements gathering as you can up front because your initial design will address only the requirements you know about, and the initial design constraints future updates.