Live data from Hacker News

Ask HN: Have you ever inherited a codebase nobody on the team could understand?

news.ycombinator.com

51–60 of 222 posts

Re: Ask HN: Have you ever inherited a codebase nobody on the team could understand?

#51

I’m a consultant and make a living saving bad projects. That’s literally why I get phone calls for work. Keep in mind that I work in high level modern languages, I’m sure there’s some crazy proprietary cpu running a robot in a Detroit factory. In any case there’s never been something that I’ve run into that I’ve not figured out. It takes time, and the hard part usually is not figuring out what it does but the weird e…

Have you used input/output proxy logging shims in these situations? They've been so invaluable to be in finding those weird edge cases that come up in 'orphaned software' projects as well as 'ancient legacy platform' migrations ;)

Re: Ask HN: Have you ever inherited a codebase nobody on the team could understand?

#52
I was hired for this exact reason at my last company and tasked with rewriting it while maintaining bit-for-bit identical output.

The company itself was in biotech (cancer diagnostics) and was relatively new, spun off from a rather well known research lab. They quickly realized that their system was incapable of scaling (or being maintained properly...) to the needs of a business.

The code itself was written primarily by one man. He was quite bright, but not really a software engineer by trade. Usual stuff; no source control, mish-mash of technologies, spaghetti code, and home grown algorithms and hardware to solve well understood problems with available solutions. New management wanted me to rewrite everything in C# (against my protests. Not because I dislike C#, only because the code dealt primarily with image analysis and hardware control/robotics.)

I began by doing exactly as you proposed; I reverse engineered every bit of code. I took extensive notes (it was hard to follow) and walked through each step from sample prep to image acquisition to analysis to result. I started writing each sub-system only after I understood how everything pieced together.

The real bitch of it was that the original developer relied on automating ImageJ for nearly all of the image analysis. As my original requirement was to not alter the results in any way, I literally rewrote large swaths of ImageJ in C#. Bugs and all.

Well, turns out ImageJ (Java) is compiled with /strictfp. C#/.NET does not support this, so my floating point results were oh so close, but not identical. This was initially a problem for management... until the CEO was replaced, along with my boss, and the new team thought the entire project was a dumb waste of money and had me build a new system from the ground up.

That system was released (successfully) early this year. I began work on it nearly five years ago now, with many detours along the way. I now work elsewhere.

Re: Ask HN: Have you ever inherited a codebase nobody on the team could understand?

#53
I inherited a project based on more than 10,000 lines of poorly commented, undocumented 8088 assembly code used on a shopping-cart attached radio+LCD panel product. It wasn't that bad an experience except the product took code updates over the air and the guy decided to use his own block checking code instead of a CRC or even a checksum on the 256 byte blocks.

The custom algorithm was a suboptimal choice as it was prone to passing badly corrupted blocks as correct. Worse, the first stage bootloader was in masked ROM so a true fix wasn't possible, only workarounds.

Re: Ask HN: Have you ever inherited a codebase nobody on the team could understand?

#54
First, you need to choose if you are going to improve it, or ultimately abandon it. That's not an easy decision. If you decide to abandon it, you need some very convincing arguments, and lots of them. Before making that decision, work with the codebase, at least until you understand it well enough. It will allow you to uncover arguments either way.

Improving a codebase is a well-known subject, so I'm not going to comment on it further.

If you decide to ultimately abandon it, you need to understand it won't happen tomorrow, and perhaps not before a few years (for example, I'm 2 years in with a codebase I decided to abandon, and it's probably going to be at least 1 more year in production). Stakeholders hate when you spend time just rewriting it for the sake of it (from their perspective).

What you want to do instead is use a strangler pattern: your new codebase should "strangle" the old one, and deliver value VERY quickly, which will convince the stakeholders it was the right choice.

First, all new features are in the new codebase. If possible, start with easy features that have as few dependencies as possible with old codebase. Any call between the two should be in a special wrapper in your new codebase, so you can start having a sense for what code will need to be rewritten, at some point.

Then, start to "strangle" the old codebase: wrap ALL calls so they go through the new codebase first. That will allow you, somewhere in the future, to cut off the old codebase part by part, and avoid the full rewrite effect, as well as quickly revert to old codebase if bugs are uncovered.

Once you have that, you can more easily identify which parts should be replaced first: performance issues, too many bugs, new features needed...

When you reach the end, it's a matter of convincing stakeholders you absolutely need to go to the last mile, with good arguments. If you can't find good arguments, you probably don't need to go to the last mile yet.

Re: Ask HN: Have you ever inherited a codebase nobody on the team could understand?

#55
"but understand the business that the code was being used in/by."

From my experience(currently on a 10MM line of code, 15 year old codebase) in software the code is the business so it's important to understand it.

First trap: "Oh, I'll just document everything."

I could say just do it, but do it for yourself. I could say that it won't be maintained over time and it'll rot and may do more harm than good when someone goes to reference it and thinks it's still an up-to-date understanding of the system.

But I'd rather just tell you to not waste your time doing it in the first place and focus on not falling into the second trap.

Second trap: Rewrite.

I could say it's easier to write new code than it is to understand it. I could tell you to be ambitious and stay up nights and weekends rewriting some view layer logic bullshit.

But I'd rather tell you that you're not as smart as you think, your solution may be more complex than the currently impenetrably complex behemoth before you, and that you should instead focus on not falling for trap 3.

Third trap: Replacement.

I could tell you there's cheaper off-the-shelf solutions available that solve the same problem; it's simply a matter of spending money and reading documentation.

But I'd rather encourage you to really, truly embrace the final realization.

Final realization: You are dumb and will never, ever understand all the complexities of this system as there are too many interdependent moving parts and other similarly complex subsystems.

And that's okay!

This is a system built over many years that consists of hundreds of thousands of tiny decisions made by hundreds of different "very smart" individuals just like you solving complex business problems giving the climate of the time it was built in.

So approach it like you would approach a beast in the wild, with caution and grace.

Heroes are at the morgue, and just make sure the thing doesn't go down.

Re: Ask HN: Have you ever inherited a codebase nobody on the team could understand?

#56

Earlier quoted context omitted.

>> My recommendation is not to try to understand the code per se, but understand the business that the code was being used in/by. I strongly agree with this. I've done at least 4 or 5 successful complete rewrites of old code bases, and I have found, rather than even 'business' the word for this might be 'context'. If you can contextualize a piece of software, it's functionality and operations, you can have a much bet…

What would you do if the codebase was actually 5 codebases absorbed from 5 different smaller companies? Assume that zero institutional knowledge about the code / business have been passed on.

You are now in the platform business.

I have to assume someone is using the software therefore there is some tribal knowledge of what it does? Otherwise this is maybe SAAS software that users use and some functionality is exposed that would allow you to begin decomposing backwards toward expected input/output. You're almost black-boxing at that point.

I will admit that I have, on very rare occasion, scream tested a piece of software running on a server that nobody would claim ownership or knowledge of either on the eng. team or within the org.

Re: Ask HN: Have you ever inherited a codebase nobody on the team could understand?

#57
tl;dr: you have probably not thought through all the ways this system is used. Make damn sure you have both rollback plans and a phased release in the works. Otherwise, all your bravery and effort will be at risk. My experience follows.

I took over a suite of flash video players/recorders for a company years ago. The variable names were exclusively 2 and 3 letter acronyms. No one on the team even knew Actionscript. Up until my hire, the various builds were all considered immutable since no one had any idea of where to start. Given their integral nature to the business, this effectively stagnated entire areas of product and business development.

What you put in our question was precisely my approach. I did my best to reverse engineer requirements by code review and sale people interviews. Once i had a confident list, i got approval and killed off what i can only assume were swaths of unused features no one knew about. Rewrote + rearchitected the entire thing from the ground up.

The release was worse than you would have expected. Turned out there were lots of these players linked outside of our website. These versions of the app were ones we had considered deprecated and were unknown to everyone in the company including the cofounding CTO. This required immediate rollbacks and hastened development to re-add features that were being used by a subset of our longest lasting clients.

While the releases themselves were rocky, the entire effort was an unquestionable success. The effort consolidated all the features into a single reusable player and opened up years of feature dev that enabled millions in new sales.

Re: Ask HN: Have you ever inherited a codebase nobody on the team could understand?

#58

Yes, on several occasions. Don't do rewrites. Morph the code towards "North." Old code has a reason for existing and being correct: It's there. From an evolutionary and survival of the fittest model, things that are out there already have a lot going for them. The ugly warts are battle scars of nasty edge cases and bugs. This is one of the main reasons I strongly value the skill of reading code in engineers. It's rar…

Good code is easy to read and change, and thus it is changed until it becomes bad code.

This is like the second law of thermodynamics applied to code.

Re: Ask HN: Have you ever inherited a codebase nobody on the team could understand?

#59
Yes, we have a few APIs written that no one on our team fully understands. All are about 3000 lines in a single file. It takes a day to make even the smallest changes (which we frequently have to do because of production incidents). All the functions take or modify maps! Its crazy. In particular we have one function that takes in 14 separate maps! And of course there are no useful comments at all.

Re: Ask HN: Have you ever inherited a codebase nobody on the team could understand?

#60
The first and most important question is, why does nobody on the team understand it? One possibility, which should not be overlooked, is that it was not quite important enough to spend money on maintaining a team of people who understand it. Just as the Big Rewrite is often not as good an option as it seems, the Big Refactor is often not a good option either, because the software may in fact not be valuable enough to justify the many hours it would take to do that.

So, first off, try to make a realistic estimate for your higher-ups of how many hours it will take to refactor this, and phrase it as "at least...[x]...and perhaps much more". It is quite possible that you will get back the answer, "it's not worth that". Then, you are in the uncomfortable position of being the Bearer of Bad News.

Depending on the ability of your upper management to accept bad news, you then either: 1) gently and politely insist to them that the situation really is this bad, or 2) start looking for an exit

But, before setting yourself up as the person who brings bad news, get a gut check from some teammates as to whether your estimates of how much would be required, are more or less on target. It takes a while for organizations to accept bad news, and you may need to let people who say "it won't be that bad" win the argument for a while, and then circle back to it in a month or so.

Post reply on HN