Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

On navigating a large codebase

blog.royalsloth.eu

Post reply on HN
Print view View original

101–110 of 139 posts

Re: On navigating a large codebase

#101

Post by bob1029 »

trias wrote: post #79
bob1029 wrote: post #51

We manage a codebase that is well over a million lines of code, and has a history dating back >5 years. One of our answers to this problem is extreme amounts of standardization. We might have 1mm LOC in platform services alone, but it is spread across 50+ types and each looks almost identical. Everything uses the same persistence mechanism, migration technique, error handling, configuration provider, etc. Dependency…

sounds like an ideology lock-in. Let's hope you never get a problem which does not fit your current architecture well, or else you'll end up spending weeks or even months solving an otherwise trivial problem. I've worked with "configuration-based implementations" and in my experience they are hard to work with (no debugging, incomplete documentation and implementation, little flexibility), require an staggering amoun…

I agree with the concern, but we have had a very long time to refine our architecture. Some would call it an ideology lock-in, I would say we solved our problem domain in a deep and meaningful way and would prefer to stick with these proven approaches. Our entire codebase was rewritten approximately 4 times before we got to the point of being confident enough to push forward with a data-driven/configuration approach.

When you are writing the same business logic hundreds of times and only 10-20 discrete things are different between each implementation, it starts to make a hell of a lot of sense to expose those things as parameters to be configured. It's simple economies of scale at this point for us. Despite our small size, we are trying to get out of a "move fast & break things" startup mindset into a more stable "lets take this to 1k customers now" mindset (we provide a B2B application in a small market, so 1k is a huge target).

For us, our company doesn't become profitable until we can scale our operations by 5-10x without any more headcount. The only thing we could come up with that would allow for this is configuration-driven techniques in which entire customer implementations can be cloned as simple JSON contracts for purposes of bootstrapping the next customer. Developers are removed from most of the product implementation process, and can focus more on core product value which is now levered hundreds of times over due to being exposed as configuration contract.

I am NOT arguing that one should seek out to build a configuration-driven system from day one. That would probably be the biggest mistake you could make. You have to already have a mostly-functional product that people already want to buy/use before you can even consider this approach. Even then, you should probably expand your target market and inject a few more use cases & rewrites before you jump over that chasm. Having a squeaky-clean domain model that addresses all potential use cases is the bare minimum prerequisite, IMO.

Re: On navigating a large codebase

#102

Post by bob1029 »

AdamCraven wrote: post #58
bob1029 wrote: post #51

We manage a codebase that is well over a million lines of code, and has a history dating back >5 years. One of our answers to this problem is extreme amounts of standardization. We might have 1mm LOC in platform services alone, but it is spread across 50+ types and each looks almost identical. Everything uses the same persistence mechanism, migration technique, error handling, configuration provider, etc. Dependency…

How was the culture of the “hive mind” developed and maintained in the organisation? I can imagine there are challenges you’ve faced to keep it working

Start small and grow carefully. Not every developer is a good fit for this type of approach and the amount of discipline we require.

We actually started looking at an approach where new hires would come in on a 6-12 month contract basis. The whole idea would be that there would be no hard feelings either way at the end if it didn't work out. If both sides felt like this was a good fit, we explore longer-term options with more benefits.

The way we do software is unconventional. We are in a very constrained environment from a security perspective. No containers, nothing can be in the cloud, all data must live on the same physical host, software delivery is tricky, etc. These constraints make the work we do somewhat unappealing to a certain crowd of developer who seeks to maximize their exposure to shiny new things.

Put differently, we use boring old technologies (with a few exceptions) and set expectations that we are going to continue to use those indefinitely. Any hopes of "mixing things up" should be reserved for future endeavors on our roadmap and personal side projects (which we encourage). I don't think any of this is unreasonable or unrealistic. We are in the business of selling software to other businesses in a sensitive market. We are not making DLC for AAA videogames.

Re: On navigating a large codebase

#103

Post by TameAntelope »

astura wrote: post #72
TameAntelope wrote: post #12

Just rewrite the dang thing! Planned obsolescence is so important for all of these reasons listed in this article. Know when its time to kill your services, and have a plan well in advance for how it'll go down, and what will take its place.

This is the worst idea I ever heard. This only "works" for companies that have unlimited VC funds to light on fire, for companies who have to actually make money, this is in no way something you can do. This is the equivalent of bulldozing your house and building another because your hot water heater broke.

It works for every company who has software they maintain, and while I’m sorry you don’t think it’s a good idea, I think the issue is more with your lack of understanding than the idea itself.

Specifically, your analogy to construction is a bad one - software is not construction, and one critical difference is the cost of rebuilding is many orders of magnitude cheaper.

When you include the reality of obsolescence into your design, you are actively anticipating and accounting for problems as they’re outlined in this article, which is always a good thing, and will always improve your planning and its outcomes.

Burying your head in the sand by expecting to never have to rebuild something is very poor project management, and not how competent software shops operate, period.

Re: On navigating a large codebase

#104

Post by TameAntelope »

barbarbar wrote: post #66
tluyben2 wrote: post #28

Earlier quoted context omitted.

For very large codebases, this is often not an option. I know of very large 'let's write Cobol mainframe to Java' projects, burning 10s of millions of euros, that were just thrown away because they could not actually get it working in the end. And this is not limited to mainframe projects; it happens with (large) more recent projects (Java/C# mostly) as well.

That raises an interesting question. Are there any such rewrites that have succeeded? I have mainly heard that it is either failures or not done.

If you are asking me if I’ve experienced a successful rewrite? Yes, numerous, and the more successful ones have happened as a result of planning.

I’ve also been part of rewrites that have gone poorly, due to a lack of planning, where the legacy software fails in unexpected and unanticipated ways, which requires a rushed attempt to fix the issue (which fails) and a subsequent rushed attempt to replace the core functionality when the fix doesn’t work (and also fails because “core” tends to be larger than you initially think).

Knowing ahead of time when software isn’t going to be useful any more isn’t really an option, it’s just an acceptance of what is already going to happen.

Re: On navigating a large codebase

#105

Post by gameswithgo »

magicalhippo wrote: post #57

This is one of the big reasons I prefer static typing. When looking at some unfamiliar code in an unfamiliar codebase, I can reason about the code much faster when I can see what functions return, and quickly go check their types out if the type is unknown. This makes me much more productive. I helped maintain a 250kLOC Python program. I came in when it already at over 200kLOC. I spent so much time , every time, just…

What little quality research there is on programming productivity, does support the idea that dynamic typing is a productivity hindrance as code bases get larger, for exactly this reason. Some studies actually have video data of the programmers at work and they can seem them having to hop around to function definitions more to figure out what they are supposed to pass in, etc

Re: On navigating a large codebase

#106

Post by zoomablemind »

Tons of complex code and noone to share the local knowledge? Isn't it a setup for a failure... yet the org is still in business, probably generating revenue.

So, I'd say the leverage is as always in understanding the dynamic and politics that often brews in places with the "monstrous" codebases.

Debug the people functions, so to speak, and it may eventually help you navigate that codebase. No one needs to be another hero, no one needs to burn-out while single-handedly fighting the beast!

Well, politics often are messy and more unpleasant than the code at hand, so we dig and rant...

In such case, I would try to limit the scope, instead of trying to learn the secret language of gods that stitched the whole system so it would make money.

My rule #1: WTF?!...But they must have had a reason for that.

Rule #2: Keep your code changes in-style; bug-free, of course, but similar idiomatically.

Rule #3: Try to do something, then try to do it together with others.

Maintaining a large codebase is not so much about tools as it is about finding ways not to do it alone.

Re: On navigating a large codebase

#107

Post by rajacombinator »

magicalhippo wrote: post #57

This is one of the big reasons I prefer static typing. When looking at some unfamiliar code in an unfamiliar codebase, I can reason about the code much faster when I can see what functions return, and quickly go check their types out if the type is unknown. This makes me much more productive. I helped maintain a 250kLOC Python program. I came in when it already at over 200kLOC. I spent so much time , every time, just…

250kLOC Python sounds scary. But that would easily be 1mLOC+ lines in Java ...

Re: On navigating a large codebase

#108

Post by trias »

bob1029 wrote: post #101
trias wrote: post #79

Earlier quoted context omitted.

sounds like an ideology lock-in. Let's hope you never get a problem which does not fit your current architecture well, or else you'll end up spending weeks or even months solving an otherwise trivial problem. I've worked with "configuration-based implementations" and in my experience they are hard to work with (no debugging, incomplete documentation and implementation, little flexibility), require an staggering amoun…

I agree with the concern, but we have had a very long time to refine our architecture. Some would call it an ideology lock-in, I would say we solved our problem domain in a deep and meaningful way and would prefer to stick with these proven approaches. Our entire codebase was rewritten approximately 4 times before we got to the point of being confident enough to push forward with a data-driven/configuration approach.…

sounds like we are working at the same company ;)

Good to see that this approach seems to be working for you, i wish you the best.

Re: On navigating a large codebase

#109

Post by Pokepokalypse »

spinningslate wrote: post #63
sydd wrote: post #32

Earlier quoted context omitted.

> Code comments? Nah. This is one of my biggest gripes. Someone (I think uncle bob) said that good code is self-documenting, which is bs in 95% of the cases. Yeah, you don't need to document the convertMinsToSecs() method, but most real life codebases are full with edge cases, shortcuts, temporary solutions, half-complete reorganizations. So people use this for writing no comments at all, whereas a few words of comme…

my priority for comments is that they should answer "why?" and "why not?" questions. Why does the method/function do it this way? Why didn't it choose that other, perhaps more obvious route? That's not necessary in every case. But it's true in a good number of them. The code alone can never tell you that - but it's often invaluable during evolution/refactoring.

Yah, and what's gross is when your team-mates criticize you for leaving comments at all. Let alone a lengthy discussion on why.

Re: On navigating a large codebase

#110

Post by phillipcarter »

astura wrote: post #65
gambiting wrote: post #42

Earlier quoted context omitted.

Speaking of IDEs - I work in video games development, huge codebase that's over a decade old, heavily templated C++ code - I've switched off the IDE "suggestions" long time ago, visual studio is just wrong about incorrect/missing code like 90% of the time. Just hit compile and read the errors, I have files that VS shows as nearly entirely wrong, squiggly lines everywhere, and yet they compile and link fine. And the o…

Yeah, visual studio is absolute and complete garbage with C++ code, it always identifies correct code as having errors, and it's not just a "big project" thing, it happens in very small projects, even "projects" that have a single file. I really don't get it... I also don't understand why intellisense doesn't update itself with the results from the compiler.

If you get the chance, when you encounter something like this that is reproducible (or at least seems obvious what's going on), you can use the Report a Problem tool and capture as many relevant diagnostics as possible. I don't work on the C++ tools team, but generally the folks working on VS are highly interested in getting detailed bug reports.
Post reply on HN