Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

On navigating a large codebase

blog.royalsloth.eu

Post reply on HN
Print view View original

51–60 of 139 posts

Re: On navigating a large codebase

#51

Post by bob1029 »

We manage a codebase that is well over a million lines of code, and has a history dating back >5 years.

One of our answers to this problem is extreme amounts of standardization. We might have 1mm LOC in platform services alone, but it is spread across 50+ types and each looks almost identical. Everything uses the same persistence mechanism, migration technique, error handling, configuration provider, etc. Dependency injection + reflection + standardization (interfaces/abstract types) is where you can get into some really powerful leverage regarding keeping things organized and sane. Ultimately we have ~8 "flavors" of thing that developers usually need to worry about.

Our end game answer is to get away from the code altogether. We are starting to view code as glue between what would ideally be configuration-based implementations and the nasty real world which must be mutated in icky ways. So, instead of writing code for a module every time you need to implement it, make it once and in a generic way, have it take a configuration object, and then expose a web UI around configuring that thing. Then, all that code is reduced to JSON being passed around. When you are dealing with pure data, you can get away with the most ridiculous things. Cloning objects, versioning, validations, relational queries, et. al. becomes trivial. If you have 1 stable domain model throughout that is 3NF or better, you can use SQL to do basically everything.

Re: On navigating a large codebase

#52

Post by jordanbeiber »

Tangentical, but relevant for complex systems and organization of them and their code:

I’ve started to look at BPMN, a thing I used to shun (bloated java enterprise junk that just slows coding down), as a way to actually help organize code.

If you have a process layer on top that describes exactly what is supposed to happen and you organize code accordingly it makes changes to complex systems easier to reason about.

I know there’s a lot more to it than this, but in my mind concepts from domain driven design, minimal service scopes and a de-coupled process layer really can help.

I guess this is why “tech” versions of bpmn-ish like systems have appeared with for example netflix conductor, uber temporal/cadence.

Split stuff up in relevant domains and describe their relationships and processes. Try to stay away from massive code bases spaning domain boundaries if possible.

No silver bullets anywhere ofc, but this is currently a topic at my employment this very moment. :)

Re: On navigating a large codebase

#53

Post by bob1029 »

bob1029 wrote: post #51

We manage a codebase that is well over a million lines of code, and has a history dating back >5 years. One of our answers to this problem is extreme amounts of standardization. We might have 1mm LOC in platform services alone, but it is spread across 50+ types and each looks almost identical. Everything uses the same persistence mechanism, migration technique, error handling, configuration provider, etc. Dependency…

Edit: One more thing I would note is that a big part of why we are able to support this codebase is because we have adopted a sort of "hive mind" developer mindset, where everyone tries to role play this ideal of a developer who would best be suited for the task. We acknowledge that our codebase is not a place for much "fun" and the best analogy I could come up with is its like doing something in a nuclear power plant control room. You just gotta do it by the book every time, and then you get to go home to a safe and happy community. It's not like we employ volunteers.

Re: On navigating a large codebase

#54

Post by varajelle »

In my opinion, apart from very tiny codebase that can fit in someone's head, there is not so much differences between medium and large codebase. You just have to use grep and the IDE to get around the parts of interests for your task at hands

Re: On navigating a large codebase

#55

Post by sgtnoodle »

davidhyde wrote: post #23

This is a great article, I can relate. I specialise in replacing large parts of codebases with code that does the same thing from a business point of view but that makes future changes cheaper to make. One thing I thing that is worth mentioning is the political aspect of this sort of work. The people in power need to be comfortable with the fact that you will be introducing risk without immediate reward. That is a to…

That's basically been my job the last five years. As soon as I joined the company (a scrappy startup of 30 people), I started refactoring large portions of mission critical code prioritized mostly by how terrified other coworkers were of touching it. In the beginning, folk were rather skeptical, but now that the company has grown by an order of magnitude, my earlier work has apparently become a topic of folklore in other parts of the company. As it usually goes, I spend most of my time these days mentoring and in meetings, but I still try to find time to refactor more fragile bits of code before they fall over completely. I encourage my team to tackle technical debt head on rather than work around it whenever possible. Far too often, folk spend more time avoiding solving a problem by patching around it, and it's usually because they're too afraid to dive in and change code that's hard to understand and therefore scary. For whatever reason, I've always had a can-do attitude when it comes to that type of work. You're paying me to get the job done, so give me an impact driver and the biggest hammer you've got. If I break something, it usually means it wasn't built strong enough to begin with. I've broken a lot of stuff over the years...

Re: On navigating a large codebase

#56

Post by atomashpolskiy »

sydd wrote: post #32
bluefirebrand wrote: post #3

This is a great article. I felt like it was describing a job I recently left, especially this piece: > It’s fine to have less experienced people working on a large system as long as they have the elders overseeing their work. In the world where senior titles are handed left and right, that is often not the case and it’s how you end up with a very fragile system that is suitable for a replacement as soon as it was bui…

> Code comments? Nah. This is one of my biggest gripes. Someone (I think uncle bob) said that good code is self-documenting, which is bs in 95% of the cases. Yeah, you don't need to document the convertMinsToSecs() method, but most real life codebases are full with edge cases, shortcuts, temporary solutions, half-complete reorganizations. So people use this for writing no comments at all, whereas a few words of comme…

While I agree to some extent, the problem with comments is that they need to be maintained in order to be helpful: code comments - updated when the code changes, general comments - when the context changes, etc. This is a work in itself: developer has to remember to do it, reviewer has to remember to look for it. In my experience, people tend to forget to do it or just don't bother, which means that someone else finds himself with a contradictory, outdated, confusing comment further down the road.

Re: On navigating a large codebase

#57

Post by magicalhippo »

This is one of the big reasons I prefer static typing.

When looking at some unfamiliar code in an unfamiliar codebase, I can reason about the code much faster when I can see what functions return, and quickly go check their types out if the type is unknown. This makes me much more productive.

I helped maintain a 250kLOC Python program. I came in when it already at over 200kLOC. I spent so much time, every time, just trying to figure out what's going on because I never knew what something returned.

Re: On navigating a large codebase

#58

Post by AdamCraven »

bob1029 wrote: post #51

We manage a codebase that is well over a million lines of code, and has a history dating back >5 years. One of our answers to this problem is extreme amounts of standardization. We might have 1mm LOC in platform services alone, but it is spread across 50+ types and each looks almost identical. Everything uses the same persistence mechanism, migration technique, error handling, configuration provider, etc. Dependency…

How was the culture of the “hive mind” developed and maintained in the organisation? I can imagine there are challenges you’ve faced to keep it working

Re: On navigating a large codebase

#59

Post by baby »

> Resist the temptation of fixing the parts that you find horrifying, because first you can’t fix it all and second you will get crushed by the complexity of the system. Mark those places down as a horrifying place to be and keep them in mind when it’s time to refactor.

Or you will run into the territory of someone else.

Re: On navigating a large codebase

#60

Post by mariusmg »

baby wrote: post #59

> Resist the temptation of fixing the parts that you find horrifying, because first you can’t fix it all and second you will get crushed by the complexity of the system. Mark those places down as a horrifying place to be and keep them in mind when it’s time to refactor. Or you will run into the territory of someone else.

Turf Wars : Code Edition
Post reply on HN