Live data from Hacker News

Use the Mikado Method to do safe changes in a complex codebase

understandlegacycode.com

71–80 of 88 posts

Re: Use the Mikado Method to do safe changes in a complex codebase

#71
Why do people promote things that are unnecessarily complicated?

I assume something about this appeals to a certain psychology. Here is the essence of the method for people who dislike rituals: pick one little thing you can succeed at. Do that thing. Repeat as necessary.

Re: Use the Mikado Method to do safe changes in a complex codebase

#72

Write tests. Most likely those 300k lines of code contain a TESST folder with 4 unit tests written by an intern who retired to become a bonsai farmer in the 1990s, and none of them pass anymore. Things become much less stressful if you have something basic telling you you're still good.

I've been working on react and react native applications professionally for over ten years, and I have never worked on a project with any kind of meaningful test coverage

People who advocate “writing tests” never admit any of the costs and difficulties of automated output checking.

As you change your codebase you will experience lots of “failures” that are not failures. You still have to burn your time investigating them.

Many checks will require elaborate mocking or other kinds of setup, that give lie to the claim that designing them is simple and straightforward.

Re: Use the Mikado Method to do safe changes in a complex codebase

#73
The more experienced I get the more I see how these simplified techniques that might be useful to a journeyman engineer in the right context can fail horribly in the wrong context.

For this particular example, the first question I have is why are we upgrading the ORM? As a codebase grows and matures, the cost of ORM change increases, and so too must the justification for upgrading it increase. Any engineer worth their salt needs to know this justification and have it mind at all times so they can apply appropriate judgment as the discovered scope increases. Let's assume now the change is justified.

The next question critical question is how do you know if you've broken anything? Right in the intro the author talks about an "untested and poorly documented codebase", but then in the example uses basic compilation as a proxy for success. I'm sorry to be harsh, but this completely hand-waves away the hard part of the problem. To have any confidence in a change like this you need to have a sense of what could go wrong and guard against those things, some of which could be subtle data corruption that could be extremely costly and hard to unwind later. This may involve logging, side-by-side testing, canary deployments, additional monitoring, static analysis and/or any other number of techniques applied based on an understanding of what the upgrade actually means under-the-hood for the ORM in question combined with an analysis of what risks that entails to the system and business/process in question. Drawing a mindmap of your refactoring plan is barely more than IntelliJ (let alone Claude Code) can already do at the click of a button.

Re: Use the Mikado Method to do safe changes in a complex codebase

#74

I've been a few times in a situation where I needed to make significant changes in a huge codebase with lot's of tests but also with a lot of corner cases, on my own. I've spent blood sweat, tears and restless evenings scrolling and ctrl-f-ing huge build and test logs to finally accomplish the task. But let's take a step back. So they assign you to get that done. You're supposed to be careful, courageous and precise…

To extend your analogy: if the house is a listed building (UK concept; apparently US equivalent is listed in National Register of Historic Places), by law you cannot just tear it down. You need to do much more work to renovate what can be done without disturbing the original structure. This obviously costs much more and is generally done by different specialists, who have harder job and hence are better paid. So the question comes back to: what kind of work do you want to do...

Re: Use the Mikado Method to do safe changes in a complex codebase

#75
I highly recommend the book "Working Effectively with Legacy Code" (Michael Feathers).

It is the best step by step guide I have ever seen to successfully work with legacy code.

Written by somebody who has been there done that many many times.

Re: Use the Mikado Method to do safe changes in a complex codebase

#76

Earlier quoted context omitted.

I've been working on react and react native applications professionally for over ten years, and I have never worked on a project with any kind of meaningful test coverage

I have not seen tests in any code base I worked on in the past 20 years. I have noticed that there is some kind of sanctimonious demeanor to quite a few people that advocate for tests (on comment boards). I find the reactions to discussions on tests fascinating because it seems to elicit very strong opinions, sort of a "do you put your shopping cart back" kind of topic, but for programmers.

I find that fascinating, because interacting with the tests in our codebase (both Python and JS) answers a _lot_ about "how is this meant to work", or "why do we have this". I won't say I do test-driven development, at least not very rigorously, but any time I am trying to make a small change in a thing I'm not 100% familiar with, it's been helpful to have tests that cover those edge cases. :)

Re: Use the Mikado Method to do safe changes in a complex codebase

#77
post #60
post #37

My favorite tool for trying scary complicated things in an unknown space is the feature flag. This works even if you have zero tests and no documentation. The only thing you need is the live production system and a way to toggle the flag at runtime. If you can ship your hypothesis along with an effectively unaltered version of prod, the ability to test things without breaking other things becomes much more feasible.…

While very powerful, I think it's worth calling out some pitfuls. A few things we've ran into - long lived feature flags that are never cleaned up (which usually cause zombie or partially dead code) - rollout drift where different environments or customers have different flags set and it's difficult to know who actually has the feature - not flagging all connected functionality (i.e. one API is missing the flag that…

It's worth noting that a feature flag mishap is what brought down Knight Capital:

https://en.wikipedia.org/wiki/Knight_Capital_Group#2012_stoc...

Re: Use the Mikado Method to do safe changes in a complex codebase

#78

Write tests. Most likely those 300k lines of code contain a TESST folder with 4 unit tests written by an intern who retired to become a bonsai farmer in the 1990s, and none of them pass anymore. Things become much less stressful if you have something basic telling you you're still good.

I've been working on react and react native applications professionally for over ten years, and I have never worked on a project with any kind of meaningful test coverage

I've checked the stats, the previous app I've worked on has 31% reported coverage and I think the actual value is higher, with coverage of most of the critical paths. But it's been a lot of work and the engineering hierarchy is supportive in adding time to manage the existing tests and test the new features.

Re: Use the Mikado Method to do safe changes in a complex codebase

#79
post #68
post #60

Earlier quoted context omitted.

While very powerful, I think it's worth calling out some pitfuls. A few things we've ran into - long lived feature flags that are never cleaned up (which usually cause zombie or partially dead code) - rollout drift where different environments or customers have different flags set and it's difficult to know who actually has the feature - not flagging all connected functionality (i.e. one API is missing the flag that…

Have them emit metrics when it's triggered. You can do a bulk "names X, Y, Z haven't used branch B in >30 days, delete?" task generator pretty easily. Un-triggered ones are also easy to catch if you force all calls to be grep-friendly (or similar), which is also an easy lint to write: unclear result? Block it, force `flag("inline constant", ...)`. Personally I've also had a lot of success requiring "expiration" dates…

Someone at work wrote a Claude skill to clean them up which seems to work fairly well but a date check would also be helpful

Re: Use the Mikado Method to do safe changes in a complex codebase

#80
post #66
post #60

Earlier quoted context omitted.

While very powerful, I think it's worth calling out some pitfuls. A few things we've ran into - long lived feature flags that are never cleaned up (which usually cause zombie or partially dead code) - rollout drift where different environments or customers have different flags set and it's difficult to know who actually has the feature - not flagging all connected functionality (i.e. one API is missing the flag that…

Yep, archiving feature flags and deleting the dead code is usually thing number 9001 on the list of priorities, so in practice most projects end up with a graveyard of them. Another issue that I've ran into a few times, is if a feature flag starts as a simple thing, but as new features get added, it evolves into a complex bifurcation of logic and many code paths become dependent on it, which can add crippling complex…

I also notice these cases tend to be missing good test coverage (at least in my experience)

I think part of the assumption is "hey there's a flag I can control if something goes wrong so manual validation is ok here" but that doesn't help when the thing is left for a period of time and everyone loses context.

Post reply on HN