How to Improve a Legacy Codebase

Oh, there's no doubt in my mind about that!

Some people may read this and extrapolate too far regarding their own situation (there's a reason this is a specialty field, it's hard stuff).

You said: I've tried explaining why we need to set up a CI server. ... In almost all cases they nod and feign interest and understanding and their eyes glaze over.

The reason you've failed to make a convincing case, I believe, is because you're talking in your language instead of theirs. Perhaps they've tried to explain to you, in their language, why they won't prioritize your CI server, and you nodded and feigned interest while your eyes glazed over.

The quote I gave you expresses your request and justification for a CI server into terms the business needs: what problem does it solve, what does it cost, how does it affect on-going costs, what are the risks of doing it and not doing it, and what impact does it have on other activities if it is done and if it is not done. This is not a "fully costed business case" or "convincing investors you need a series A". If you've given any thought at all to why you want a CI server beyond "I want it" you should have no problem filling in the blanks in my quote. And if you haven't bothered to think that much about it, your business is doing the right thing by giving your requests a low priority, because they shouldn't give your ideas any more attention than you're giving them yourself.

500K lines is 'small' by our standards and if we are not moving within two weeks that translates into one very unhappy customer. That's something a typical team of 5 to 10 people has produced in a few years.

Note that I wrote 'incremental' and 'starting'. That doesn't mean the job is finished at that point in time. But we should have a very solid grasp of the situation, which parts are bleeding the hardest and what needs to be done to begin to plug those holes. That the whole thing in the end can become a multi-year project is obvious, we're not miracle workers, merely hard workers.

In a way the size of the codebase is not even relevant. What is most important is that you get the whole team and the management aligned behind a single purpose and then to follow through on that. Those first couple of weeks are crucial, they are tremendously hard work even for a seasoned team that has worked together on jobs like these several multiple times.

The one case I wrote about here was roughly that size (so small by my standards), within 30 days the situation was under control. We're now two years later and they are still working on the project but what was done in that short period is the foundation they are still using today.

If a project is much larger than that then obviously it will take more time. Just the discovery process can take a few weeks to months, but in that case I would recommend to split the project up into several smaller ones that can be operated on independently with 'frozen interfaces' where-ever they can be found.

That way you can parallelize a good part of the effort without stepping on each others toes all the time.

The problem is not that you can't tackle big IT projects well. The problem is that big IT projects translate into big budgets and that in turn attracts all kinds of overhead that does not contribute to the end result.

If you strip away that overhead you can do a lot with a (relatively) small crew.

If you're going to tackle a code base in excess of something 10 M loc in this way you will again run into all kinds of roadblocks. For those situations it would likely pay off to spend a few months on the plan of attack alone.

If a project that large came my way I would refuse, it would tie us down for way too long.

But that's out of scope for the article afaic, we're talking about medium to large project, say 50 manyears worth of original work that has become unmaintainable for some reason or other (mass walk-out, technical debt out of control or something to that effect).

If those are 'very very small projects' by your standards than so be it.

And that's the crux of the problem. The business invariably mistakenly believes that piling more features onto the steaming pile of crap that is the codebase is the better solution. Add on to that that some mid-level PM promised feature X to the C-level in M months, where M is such short notice even a engineering team with cloning and time machines would be short-staffed, and was chosen without even asking the engineering staff what their estimate of such work would be.

To the business, the short term gains of good engineering practices are essentially zero. The next feature is non-zero. The long-term is never considered.

I've had multiple PMs balk at estimates I've given them. "How could internationalizing the entire product take so long? We just need to add a few translations!" No, we need to add support for having translations at all, we need to dig ourselves out from under enough of our own crap to even add that support, we need to figure out what text actually exists, and needs translating, actually add those translations, and we need to survey and double-check a whole host of non-text assets because you mistakenly believe that "internationalization" only applies to text. Next comes the conversation about "wait, you can't just magic me a list of strings that need translating? I need that for the translators tomorrow!" No, they're mixed in with all the other strings that don't need translating, like the hard-coded IPv5 address of the gremlin that lives in the boiler room eating our stack traces.

Then, later, we'll lose a week of time because the translation files that engineering provided were turned into Word documents by PMs. One word doc, with every string from every team, and then those Word docs got translated. So now we have French.docx, but that of course only has the French. So now engineers are learning enough French to map the French back to the English so they know what translations correspond to what messages.