Live data from Hacker News

Ask HN: Have you ever inherited a codebase nobody on the team could understand?

news.ycombinator.com

31–40 of 222 posts

Re: Ask HN: Have you ever inherited a codebase nobody on the team could understand?

#31
Yes, it was a mega legacy codebase written by a single person over the span of a decade and was extremely “job secured”. It had Perl scripts that would system call to php scripts that would in turn do a curl request to another http perl script that would system call another php script that would output HTML, which then would get parsed by the calling scripts several ways. That was just one place. There were lots and lots of these problems.

Our team was handed the project to rewrite it. It was a secret project and we were very careful. We didn’t want to spook the original developer.

It was a lot of hair pulling and tracking of the code. Lots of gruelling work.

Re: Ask HN: Have you ever inherited a codebase nobody on the team could understand?

#32
Yes, frequently. A couple of highlights:

PostgreSQL allows C extensions. A function which used to work was now segfaulting occasionally so needed to be fixed. Original author was gone, nobody really understood the PostgreSQL extension system (which involves a ton of macros.) No version control, no useful comments, no spec. I read the code (the core of the logic was straight-forward C, it was just the interface to PostgreSQL that was hard to understand) and wrote a new version in Python (another language for which PostgreSQL supports extension functions.) That version was too slow, by two orders of magnitude. Went back to the C implementation. Did a little light refactoring to separate the core logic from the PostgreSQL interface. Then wrote a test harness program in C to drive call only the inner function. Wrote unit tests until I reproduced the segfault. Fixed the C code and tested it with my test harness. The PostgreSQL wrapper just called the (now correct) inner function so it now worked too. Checked the fixed code and unit tests into version control.

A PhD (no longer with the company) had fit some simple neural net models in R. He'd written his own code for this because, according to him, the standard packages in R didn't support a few of the bells and whistles he'd wanted like ReLU activation. Not only was the code in-house, but the models themselves were saved as serialized R objects. No specifications for any of this stuff. Apparently the company had been using this code to score medium size databases for several years. When we wanted to scale up to a much larger database (approximately 30 billion rows) the problems with the R implementation became apparent. Fairly slow, high memory usage, worse yet a slow memory leak on large jobs, and worst of all occasional silent crashes where it would simply stop and exit with a successful status of 0 and no error message. This time I took the approach of reading the code and de-serialized R objects. I re-wrote the implementation in Python using numpy arrays and wrote a small R program to read the serialized models in the .RData format and emit a cleaned up JSON object that could be easily read from Python. Luckily I didn't have to port any of the cross-validation/training/optimization stuff; just the prediction part. That meant 80% of the code could be safely ignored. However, the devil was in all the one-off special cases in the serialized R objects, many of which had behavior different from default. I would test by comparing the predictions by the two programs on small batches of a million. These predictions were floating point numbers between 0 and 1 so when both programs agreed to within 1e-5 for all million I knew it was correct. It took a week to track down all the special cases, though, and the special cases made it impossible to just use a standard neuralnet library like Keras. (We already had several Keras models in production and a set of tools to manage them; that would have been easy for us.) Proprietary code begets proprietary code I guess. At least the new implementation was much faster and didn't leak memory so could deal with the whole database in a single long running process. I pitched the idea of re-training all models from scratch in Keras using our more modern tools but management wanted 100% backwards compatibility and to preserve the value-add of the PhD.

Re: Ask HN: Have you ever inherited a codebase nobody on the team could understand?

#33

That is essentially what I was hired out of school to do. I walked into a massive heap of ASP.NET (with VB) and MSSQL stored procedures that didn't really work at all, and had been through the wringer of a few cut-rate outsourcing groups. I struggled along with it for a few months figuring out how it was supposed to work and trying to duck-tape it together, doing a lot of support with customers that were trying to us…

The problem with going this route is you have to sacrifice personal time for it. Now in your case you taught yourself something new, but that's not always going to work.

I do agree with your assessment that it is much easier to drop a working thing in people's laps and get buy-in than ask for permission. This is also what causes pork barrelling in projects.

Re: Ask HN: Have you ever inherited a codebase nobody on the team could understand?

#34
Usually it's a big single file of spaghetti, which is the bigger issue... I tend to try to separate logic branches (if/else) into separate functions. In addition to this, often a big if, with no else that just returns, I'll reverse the logic.

In then end, just like eating an elephant... one bite at a time. Eventually you'll have everything broken enough into separate functions that you then understand the whole better and can cleanly rewrite the whole thing.

Re: Ask HN: Have you ever inherited a codebase nobody on the team could understand?

#35
I once inherited a large enterprise system I had to build a full working development environment for and then learn the code base and help bring a team up to speed.

The product was essentially made up of 5 code bases in a single repo that built and deployed 5 different executables that worked together. That itself wasn't so bad, it was mostly modern C# and I found it a decent code base to work on.

What was bad was that one of the modules that we were expected to support was written entirely in VB6. The last stable release of VB6 was in 1998. I couldn't just download something and install it and work with the VB6. It turns out that the accepted solution according to stackoverflow, and this is the conclusion I came to independently too, was to go on EBay and try to buy a copy of software/compiler/IDE/whatever and then to even get it to install and work you need to do all kinds of things like turning off certain keys in the registry before installation and enabling them again afterwards then doing a bunch of other modifications to make sure that it actually works when run in compatibly mode with Windows XP.

Our official line was we went from supporting 5/5 modules to 4/5 modules.

Fun times.

Re: Ask HN: Have you ever inherited a codebase nobody on the team could understand?

#36
Yes, on several occasions.

Don't do rewrites. Morph the code towards "North." Old code has a reason for existing and being correct: It's there. From an evolutionary and survival of the fittest model, things that are out there already have a lot going for them. The ugly warts are battle scars of nasty edge cases and bugs.

This is one of the main reasons I strongly value the skill of reading code in engineers. It's rare and so important.

Re: Ask HN: Have you ever inherited a codebase nobody on the team could understand?

#37

I once inherited a large enterprise system I had to build a full working development environment for and then learn the code base and help bring a team up to speed. The product was essentially made up of 5 code bases in a single repo that built and deployed 5 different executables that worked together. That itself wasn't so bad, it was mostly modern C# and I found it a decent code base to work on. What was bad was th…

Do they not make Virtual Machines where you come from? Or am I missing something?

Re: Ask HN: Have you ever inherited a codebase nobody on the team could understand?

#40
I once inherited a compiled executable.

And source code that was demonstrably older than the binary.

It was written by a contractor, who was blackmailing us for the up-to-date source code.

In an FDA-regulated industry.

How did we deal with it?

We pretending nothing was wrong, and prayed that no show-stoppers would happen. We begged for permission to rewrite, but it was deemed to be too expensive.

And I left that company as soon as I could.

Post reply on HN