Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

On navigating a large codebase

blog.royalsloth.eu

Post reply on HN
Print view View original

91–100 of 139 posts

Re: On navigating a large codebase

#91

Post by coderdd »

sillysaurusx wrote: post #24

I navigate codebases by cat'ing all the files together (prefixed with filename) and piping it into vim. I was shocked how much I learn by this seemingly-horrible technique. For example, the python files that actually get deployed are often quite different from the ones in source control. For tensorflow, at least. I regularly read 1M+ lines of code this way. Not an exaggeration; vim scales, nothing else does.

I have great success using Zoekt, a Google codesearch reincarnation, for instant grepping.

Re: On navigating a large codebase

#92

Post by tetha »

wwwigham wrote: post #81

The article mentions the importance of comments and documentation inline in code. I tend to agree - well-written code is great and all, but a good comment can bring in context external to the code and make _why_ code is what it is more clear to future readers. And reviewers. Comments explaining _what_ code does largely aren't needed - that's evident from usage. But the _why_? Some people would say code which can't be…

To me, there is also the question of change frequency. Of course, it's a guesstimate all the time, but things that look to be rarely-changing (and usually ends up being high-impact) deserve over-documentation. Something like our AWS VPC, DNS and DHCP setups and their expected change modes and their effects are very well documented. It rarely changes and if it breaks, everything breaks. The fundamentals of our disk setups and disk encryption in ansible is carefully documented because if that breaks, things will go hairy.

In those cases, having a lot of documentation, speeds up changes because you can store months and years of deliberation and decisions in these comments.

Re: On navigating a large codebase

#93

Post by wiredfool »

spinningslate wrote: post #63
sydd wrote: post #32

Earlier quoted context omitted.

> Code comments? Nah. This is one of my biggest gripes. Someone (I think uncle bob) said that good code is self-documenting, which is bs in 95% of the cases. Yeah, you don't need to document the convertMinsToSecs() method, but most real life codebases are full with edge cases, shortcuts, temporary solutions, half-complete reorganizations. So people use this for writing no comments at all, whereas a few words of comme…

my priority for comments is that they should answer "why?" and "why not?" questions. Why does the method/function do it this way? Why didn't it choose that other, perhaps more obvious route? That's not necessary in every case. But it's true in a good number of them. The code alone can never tell you that - but it's often invaluable during evolution/refactoring.

Why does this code exist?

What are you working around?

What are the assumptions? What limitations?

If it's complicated enough that I'm only understanding it because of the context of the last week, we need the comments. Anything that can speed the reverse engineering in 6 months when it breaks is helpful, because then you can quickly decide that we got different input or if we missed an edge case or whatever.

Re: On navigating a large codebase

#94

Post by spinny »

mtzet wrote: post #16

The advice about using both grep /and/ the IDE is very good. Often they are framed as in opposition to each other, but in reality they're just tools. IDE's are great when they work, but it's entirely possible to make it confused. I keep hearing to get better IDE's, especially from Java developers who seem to have nicer IDE's than us C++ schmucks, but even the best IDE will not save you when your program is really an…

Visual Studio Code is nice in this aspect. When you use the terminal window, the output is parsed and you can ctrl+click on a filename or (filename:line) to jump to it

Re: On navigating a large codebase

#95

Post by KajMagnus »

sydd wrote: post #32
bluefirebrand wrote: post #3

This is a great article. I felt like it was describing a job I recently left, especially this piece: > It’s fine to have less experienced people working on a large system as long as they have the elders overseeing their work. In the world where senior titles are handed left and right, that is often not the case and it’s how you end up with a very fragile system that is suitable for a replacement as soon as it was bui…

> Code comments? Nah. This is one of my biggest gripes. Someone (I think uncle bob) said that good code is self-documenting, which is bs in 95% of the cases. Yeah, you don't need to document the convertMinsToSecs() method, but most real life codebases are full with edge cases, shortcuts, temporary solutions, half-complete reorganizations. So people use this for writing no comments at all, whereas a few words of comme…

> Someone (I think uncle bob) said that good code is self-documenting, which is bs in 95% of the cases

Agreed. For example, comments about Why-do-this, and Why-Not-do-that can be necessary, even if the code shows what happens.

Imagine you're in a taxi, and it suddenly takes the wrong turn, now instead heading towards Surprise-City. Then — you know what is happening. You're going to Surprise-City.

But would't you also want to know Why?

So then it's nice if the taxi driver explains Why: "I buy milk to kitten."

I think the 'Linux kernel coding style' explains comments pretty well:

https://www.kernel.org/doc/html/v4.10/process/coding-style.h...

Re: On navigating a large codebase

#96

Post by minusf »

sillysaurusx wrote: post #45
Gehinnn wrote: post #41

Earlier quoted context omitted.

If you read/scan 10 lines a second, you still need over 24 hours non-stop to read a 1M+ code base. I doubt a random file ordering is helpful! Especially if you lose code navigation features like "go to definition".

You'd be surprised. Suppose I want to "go to definition" for a class named Saver. /^class Saver\> 19 times out of 20, this works. It's also instant; my vim will likely get me there faster than your IDE's go to definition functionality. (Looking at you, pycharm!) Here's my flow. >>> import tensorflow as tf >>> tf.train.Saver >>> from tensorflow.python.training import saver >>> saver Then I open /usr/local/lib/python3.…

i do something similar but with tags and without the merging. vim's tag navigation is very powerful. split a window with the target tag and see side by side the class and its descendant for example.

i use `venv`s for every project inside the project folder so uctags also generates tags for all libraries installed and i can "drill all the way up" to classes and definitions.

it's also possible to have the system libraries show up in the tag database, it's just a matter of telling uctags which path's, files to include/exclude or alternatively use another tag file for that, vim can use multiple tag files.

the real pain point is keeping the tag file update. gutentags makes this a bit more easy for me.

Re: On navigating a large codebase

#97

Post by sillysaurusx »

minusf wrote: post #96
sillysaurusx wrote: post #45

Earlier quoted context omitted.

You'd be surprised. Suppose I want to "go to definition" for a class named Saver. /^class Saver\> 19 times out of 20, this works. It's also instant; my vim will likely get me there faster than your IDE's go to definition functionality. (Looking at you, pycharm!) Here's my flow. >>> import tensorflow as tf >>> tf.train.Saver >>> from tensorflow.python.training import saver >>> saver Then I open /usr/local/lib/python3.…

i do something similar but with tags and without the merging. vim's tag navigation is very powerful. split a window with the target tag and see side by side the class and its descendant for example. i use `venv`s for every project inside the project folder so uctags also generates tags for all libraries installed and i can "drill all the way up" to classes and definitions. it's also possible to have the system librar…

i use `venv`s for every project inside the project folder so uctags also generates tags for all libraries installed and i can "drill all the way up" to classes and definitions.

it's also possible to have the system libraries show up in the tag database, it's just a matter of telling uctags which path's, files to include/exclude or alternatively use another tag file for that, vim can use multiple tag files.

Oh?

Yours is the first system I've found that has this very important feature -- the whole reason I do it my way is because I can dill down into the actual installed libraries, whereas IDEs almost always fail. (It's hit or miss. Yeah, theoretically you can configure the IDE properly if you spend your life becoming an IDE master and have a 96-core workstation, but it never seems to "just work.")

If you ever do a writeup of how precisely you've set up your environment, do ping me! I'm https://twitter.com/theshawwn. I'd be very interested and would happily retweet it.

Re: On navigating a large codebase

#98

Post by chromatin »

OpenGrok, while an older tool, is web based and a very nice way to explore large codebases; and it is multi language as well [2]

To get you started, here are public instance with source code for Illumos [3] and multiple BSDs [4]. Used to have a Linux one but cannot find atm.

[1] https://oracle.github.io/opengrok/ [2] https://github.com/oracle/opengrok/wiki/Supported-Languages-... [3] http://src.illumos.org/source/ [4] http://bxr.su/

Re: On navigating a large codebase

#100

Post by disgruntledphd2 »

sydd wrote: post #86
disgruntledphd2 wrote: post #74

Earlier quoted context omitted.

So, the advice is bad, as most people (including myself, no doubt), will write mediocre code, just by the shape of the distribution (assuming it's normally distributed, which is a strong assumption, but without data it's probably reasonable). Advice that relies on people caring about their craft/having the skills to do the work well doesn't scale, so it's bad advice where those things aren't true.

Also in lots of cases you are in a hurry to meet that deadline that compromises code quality. Better to leave a comment in this case than nothing

Yeah, especially if you do something weird. People have often done weird strange stuff that made sense when I finally figured out the reason.

Time pressure (and particularly with contractors) can lead to some horrific long-term burdens of maintenance.

Post reply on HN