Live data from Hacker News

Ask HN: How to be productive with big existing code base

news.ycombinator.com

61–70 of 187 posts

Re: Ask HN: How to be productive with big existing code base

#61
post #12

Do they have good test coverage? That's key. If they don't, start with that.

I've discovered that this advice doesn't work for me because I really prefer to see the software being used in a real world scenario than in tests. This is also why I hate writing tests for something I didn't develop.

So instead, I usually try to setup the dev stack and walk through multiple use cases of the product, usually narrowed down to the area I would be working on. I would make sure my stack is so that I can iterate really really fast (I've a few quirks that I tend to use like weird aliases etc etc) and all this while also simultaneously working on small/minor bug fixes. I do this for atleast 3-4 weeks atleast, then I get a feel for the code and where things are. After that, I personally feel confident enough to take on bigger projects on that codebase.

Has worked out well for 3 large codebases at 3 different companies so far.

Re: Ask HN: How to be productive with big existing code base

#62
Step 1: investigate. find out where the program is breaking and/or delivering unacceptable performance. This means asking stakeholders "is anything broken? What is wrong with the app? what do you want to change?", reviewing exceptions, and crash logs. Keep in mind that often times stakeholders don't know what they want - determine the business objective and then work from that. Determine where the broken code is, and write narrative comments about what you think it does. If there is documentation, read it. There often isn't any documentation, and often times the comments are useless on a good day. Resign yourself to playing a combination garbage man/forensic psychologist for the next few months.

Step 2: triage. determine which broken parts you can get away with leaving alone for the short term, and which parts need immediate attention.

Step 3: Fix the most critical broken parts incrementally. If there are no tests (there are never tests...), write a test for each block of code you modify. Avoid wholesale redesigns if possible. make sure that you write tests. Try to avoid getting mad about the previous person's style - focus on getting things working to a borderline acceptable level, writing comments to explain your decisions so you or someone else has a frame of reference. The goal of doing this is to buy yourself time to clean the entire thing up.

Step 4: Once the app is working at a baseline acceptable level, examine the codebase and determine which areas (if any) require redesigns, and determine the cost/benefit of each redesign, based on what the stakeholders need, want and expect. If any redesign is necessary, negotiate with stakeholders to buy time for it - your bargaining chip should be an additional feature or two that the previous guy shat the bed on. Basic criteria for a redesign: is the current design impossible to understand? Does the current design impose unacceptable costs in terms of performance or development time? if yes to either, a redesign is probably worthwhile.

Re: Ask HN: How to be productive with big existing code base

#63

Document everything as you explore it. I'm an advocate for literate programming but accept it's not going to be accepted by most organizations. So I use it as a personal tool. Tools: emacs, org mode, org babel. Create a parallel directory structure, hypothetical project: ./src ./project/src/main.js ./project/src/some-file.js Create a new directory structure with one org file per source file and one index org file: ./…

As someone who would rather read comments than code, I like the idea of "literate programming". But I was expecting you to be taking notes about each function, not documenting the file structure. What's the goal of breaking the file down like this? You don't try to maintain this when you change the code, right? So it's just a one-time familiarization with the code files? But why do I need to note down "This section o…

I threw that post together in about 5 minutes at work and never came back to it.

I do describe the what and why more, but I start with the code structure because that's what I've been handed and need to understand. I also work in embedded systems where, generally, the code call tree is acyclic and sticking with this format works well (each file is often its own module with clearly defined, if not clearly documented, interfaces for the outside). If I weren't dealing with these systems I'd need to reconsider the structure.

It's not just "this section has structs". I'd start with that:

  ** Structs
  #+NAME: structs
  #+BEGIN_SRC c
  // all the struct defs
  #+END_SRC
then:

  ** Structs
  #+NAME: structs
  #+BEGIN_SRC c :now yes
    >
    // rest
  #+END_SRC
  *** msg123
  This struct holds messages of type 123. Here's the spec for it [some link].
  Here's a list of each component and their acceptable values:
  - type :: stored in the lower 5-bits of the first 16-bit word, should be 123
  - timetag :: stored in the second 16-bit word, represents time since midnight
    in seconds.
  #+NAME: msg123
  #+BEGIN_SRC c
    // code
  #+END_SRC
If the structs are straightforward (think a standard quick-and-dirty llnode definition), I won't bother breaking it down because it's clear (for me). If I'm communicating it to a new developer, maybe I write more.

While I don't actually develop from this code (except for personal projects), I do a sanity check. I attempt to tangle the code (generate the source output from the org files) and run a git diff. If it shows any non-whitespace differences, then I accidentally altered a line I didn't mean to. Which means my documentation will be wrong.

Re: Ask HN: How to be productive with big existing code base

#64

This is super specific to each project but here things that worked for me in previous projects. Two assumptions: You plan to work on this longer-term (not a 1month project stint) and there are things worth improving (eg barely used legacy app might not be worth your time) #1 Get the team on board if there are multiple people you need their buy-in and support for whatever approaches you want to do #2 Plan for "health…

Disclaimer: I don't know anything about managing a team, this advice is more for a solo developer or a developer joining a team.

#2 is what I use. After one/two hundred tiny commits (even if it's sometimes just fixing grammar in comments) you'll feel more at home. And because they're small you can achieve that within a few weeks.

This requires any way of telling that things still work. Sometimes you'll be able to do that because the transformation is an identity given the language definition, but for other cases I'd encourage #7. Having someone more senior (with respect to the codebase) review changes helps (this is what I mostly used), but I understand you don't have that option.

Re: Ask HN: How to be productive with big existing code base

#65
post #8
post #6

My #1 rule for existing codebases: Just because you wouldn't have done it the way they did doesn't mean they did it wrong. I think it's developer nature to look at a huge pile of code that someone else wrote and immediately think: "This is a pile of crap. I can do better, so the first thing to do is rewrite all of this, my way (which just so happens to be _The Right Way_)." Figure out what you're trying to do, and wh…

"This is a pile of crap. I can do better," I'll take this one step further and say: if you think this, you're unqualified for the position. You are an amateur.

If you always think this you’re probably an amateur, but if you seen good code that wasn’t yours, then you might have a sense for things. Of course that requires that you read a lot of code.

Re: Ask HN: How to be productive with big existing code base

#66
post #43
post #22

Earlier quoted context omitted.

There's a term for this, Chesterton's Fence: https://en.wikipedia.org/wiki/Wikipedia:Chesterton%27s_fence > let us say, for the sake of simplicity, a fence or gate erected across a road. The more modern type of reformer goes gaily up to it and says, "I don't see the use of this; let us clear it away." To which the more intelligent type of reformer will do well to answer: "If you don't see the use of it, I certainly w…

The counterpoint to Chesterton's Fence is the psychology experiment with the monkeys, the stairs and the banana. This is long but very pertinent to the "that's just how we do things around here" attitude: > This human behavior of not challenging assumptions reminds me of an experiment psychologists performed years ago. They started with a cage containing five monkeys. Inside the cage, they hung a banana on a string w…

I don't see how that's a counterpoint. Chesterton doesn't say "don't change it, that's how it is and shall be." He's saying, "Question why it's present before you attempt to change it."

Looking around, there were no obvious livestock or other reasons for the fence. But perhaps they're only there six months out of the year. Ok, make the fence a gate, and not a fence, so it can be opened more easily. Maybe it's no longer used for livestock, ok remove it altogether. But if it is used for livestock (but they were just over the hill that day), and the fence is removed, an error has been made (it existed for a reason) and the livestock can now wander beyond the lands they're supposed to be on.

Re: Ask HN: How to be productive with big existing code base

#67
post #12

Do they have good test coverage? That's key. If they don't, start with that.

One thing to consider: large code bases are large. To get any kind of useful test coverage it may take you a very long time. While you are doing that, you won't be visibly making any progress as seen from other parts of the company. I often describe it as a kind of horizon -- you can do what you want until you get to a time horizon. After that, the rest of the company feels like they have lost sight of you. This causes them to panic and they will usually alter your priorities dramatically. My rule of thumb is that your time horizon is about 2 weeks. If you don't make visible progress in that time frame, your project takes on more and more risk as you sail away.

Instead, try to find as much low hanging fruit as you can find. Fix bugs. Add small features. Address nagging complaints that have been around for a long time. At the same time, start filling out your test framework (and also beef up your build process). Keep pairing refactoring and janitorial tasks with "work that pays the bills".

One last thing. Try to identify areas where your users/paying customers/stakeholders think, "This shouldn't be hard" and cross reference that with areas that are hard due to code complexity. Make sure to pick some of that work so that you can make the tests in that area robust. Once that's done, start refactoring so that you can do those requests very quickly. Once you do that don't forget to advertise your success! "Remember how much of a pain X used to be? Now we can do it quickly because we worked hard on fixing the problems". This will give you much needed political capital that you will spend investing on improving other more tricky areas of the code.

Re: Ask HN: How to be productive with big existing code base

#68
focus on outcomes. have an objective like better monitoring, better perf, better code coverage, and use that as your guiding principal. make sure any change benefits the customer or your team. avoid refactoring just to modernize, every change is a potential regression. and what is in style now will be legacy in due time.

once you decide on the objective, try to divide components up into interfaces so you can test and refactor a component with reduced side effects.

set a goal for each objective. like improve coverage 5% a month or no net neg.

long story short, be methodical and focus on outcomes

Re: Ask HN: How to be productive with big existing code base

#69
You haven't said much about your role here, but let's assume that programming figures prominently. Your immediate problem is that you will be given tasks that involve making changes to a codebase that you don't understand very well. There are many different ways to understand a codebase, and to be effective, you will need to learn some of all of them.

Firstly, there is the purpose of the system, which means getting to know what its users want from it and how they use it to achieve that. I put that first, because everything else follows from it, but that does not mean that you have to know all there is to know about that aspect before tackling anything else.

You will need to learn about your environment's process: whatever is used for task assignment, scheduling and tracking; version control; building; testing and verification, inspections, test setup and execution; integration and deployment. Of these, you will need to know how to get the source code and test any changes you make, before you can do any programming, and a significant landmark in getting to know the system is when, given nothing but a backup of the source and configuration files, you could resurrect it.

The more you know about the architecture, the better - it is the first step in understanding how the system meets (or fails to meet) the users' needs. The architecture often imposes requirements and constraints on how you approach completing a given task.

Understanding how everything works at the code level would be a desirable goal, but not one that can be achieved quickly (if at all), so you will have to be guided by what you need to understand in order to do do your assigned tasks.

It is also useful to know who knows what among the people you will be working with.

Re: Ask HN: How to be productive with big existing code base

#70
post #6

My #1 rule for existing codebases: Just because you wouldn't have done it the way they did doesn't mean they did it wrong. I think it's developer nature to look at a huge pile of code that someone else wrote and immediately think: "This is a pile of crap. I can do better, so the first thing to do is rewrite all of this, my way (which just so happens to be _The Right Way_)." Figure out what you're trying to do, and wh…

"When you see something that's old and it's been there for a long time and it's working, don't laugh at it."

(From a not-really-related yet quite interesting talk I watched a while back: https://www.youtube.com/watch?v=TQB0KK2rxcw#t=31m0s --- also featured on HN at https://news.ycombinator.com/item?id=18430512)

Post reply on HN