Live data from Hacker News

Code search is hard

blog.val.town

111–120 of 164 posts

Re: Code search is hard

#111
post #49

Earlier quoted context omitted.

> If you ever leave you can use Livegrep, which was based on code-search work done at Google. If I’ve learned anything from the fainting spells that I-work-at-X have over their internal tools on HN: no, whatever the public/OSS variant is always a mere shadow of the real thing .

I suspect you're being sarcastic - but can confirm that being nearly two years out of Amazon, I still miss its in-house CD system nearly every day. I've actively looked around for OSS replacements and very few come anywhere close. (I would be _delighted_ for someone to "Umm actually" me by providing a great product!)

Is it true that teams don't do branches in source control at all? Just publishing a CR?

Re: Code search is hard

#112

Basic code searching skills seems like something new developers are never explicitly taught, but which is an absolutely crucial skill to build early on. I guess the knowledge progression I would recommend would look something kind this: - Learning about Ctrl+F, which works basically everywhere. - Transitioning to ripgrep https://github.com/BurntSushi/ripgrep - I wouldn't even call this optional, it's truly an incredi…

Also Github is a fantastic tool for searching code across repos, ones you may not even have cloned yet! Either public ones or org ones.

Re: Code search is hard

#113

OpenGrok ( https://github.com/oracle/opengrok ) is a wonderful tool to search a codebase. It runs on-prem and handles lots of popular programming languages.

I fully agree with you there, OpenGrok is a wonderful, oudated-looking and feeling but lightning fast code search engine!

Re: Code search is hard

#114

Basic code searching skills seems like something new developers are never explicitly taught, but which is an absolutely crucial skill to build early on. I guess the knowledge progression I would recommend would look something kind this: - Learning about Ctrl+F, which works basically everywhere. - Transitioning to ripgrep https://github.com/BurntSushi/ripgrep - I wouldn't even call this optional, it's truly an incredi…

Also Github is a fantastic tool for searching code across repos, ones you may not even have cloned yet! Either public ones or org ones.

yeah, I particularly like the combo regex + path: or lang:

Re: Code search is hard

#115
Why am I not seeing anything here about ctags[0] or cscope[1]? Are they that out of fashion? cscope language comprehension appears limited to C/C++ and Java, but “ctags” (I think I use “uctags” atm) language support is quite broad and ubiquitous…

[0] https://en.wikipedia.org/wiki/Ctags

[1] https://en.wikipedia.org/wiki/Cscope

Re: Code search is hard

#116
post #7

It indeed is hard, and a good code search platform makes life so much easier. If I ever leave Google, the internal code search is for sure going to be the thing I miss the most. It's so well integrated into how everything else works (blaze target finding, guice bindings etc), I can't imagine my life without it. I remember to appreciate it even more every time I use Github's search. Not that it's bad, it's just inhere…

The guide bindings layer thing is nice, but its UI could be improved. I wish I could directly find for providers/usages from the search box.

Re: Code search is hard

#117
It's why IDE and developer tool builders have long had the insight that in order to do code search properly, you need to open up the compiler platform as a lot of what you need to do boils down to reconstructing the exact same internal representations that a compiler would use. And of course good code search is the basis for refactoring support, auto completion, and other common IDE features.

Easier said then done of course as tools are often an afterthought for compiler builders. Even Jetbrains made this mistake with Kotlin initially, which is something they are partially rectifying with Kotlin 2.0 now to make it easier to support things like incremental compilation. The Rust community had this insight as well with a big effort a few years ago to make Rust more IDE friendly.

IBM actually nailed this with Eclipse back in the day and that hasn't really been matched since then. Intellij never even got close to this being 2-3 orders of magnitudes slower. We're talking seconds vs. milliseconds here. Eclipse had a blazing fast incremental compiler for Java that could even partially compile code in the presence of syntax errors. The IDEs representation of that code was hooked into that compiler.

With Eclipse, you could introduce a typo and break part of your code and watch the IDE mark all the files that now had issues across your code base getting red squiggles instantly. Fix the typo and the squiggles went away, also without any delay.

That's only possible if you have a mapping between those files and your syntax tree, which is exactly what Eclipse was doing because it was hooked into the incremental compiler.

Intellij was never able to do this, it will actively lie to you about things being fine/not fine until you rebuild your code and it will show phantom errors a lot when it's internal state gets out of sync with what's on disk. It often requires full rebuilds to fix this. If you run something, there's a several second lag while it compiles things. Reason: the IDE internal state is calculated separately from the compiler and this gets out of sync easily. When you run something, it has to compile your code because it hasn't been compiled yet. That's often when you find out the IDE was lying to you about things being ready to run.

With Eclipse all this was instantly and unambiguous because it shared the internal state with the compiler. If it compiled, your IDE would be error free, if it didn't it wouldn't be. And it compiled incrementally and really quickly so you would know instantly. It had many flaws and annoying bugs but that's a feature I miss.

Re: Code search is hard

#118
post #115

Why am I not seeing anything here about ctags[0] or cscope[1]? Are they that out of fashion? cscope language comprehension appears limited to C/C++ and Java, but “ctags” (I think I use “uctags” atm) language support is quite broad and ubiquitous… [0] https://en.wikipedia.org/wiki/Ctags [1] https://en.wikipedia.org/wiki/Cscope

exactly THIS the only problem with `cscope` is that for modern c++ based code-bases it is woefully inadequate. for plain / vanilla c based code-bases f.e. linux-kernel etc. it is just _excellent_

language-servers using clangd/ccls/... are definitely useful, but quite resource heavy. for example, each of these tools seem to starting new threads per file (!) and there are no knobs to not do that. i don't really understand this rationale at all. yes, i have seen this exact behavior with both clangd and ccls. oftentimes, the memory in these processes balloon to some godawful numbers (more with clangd than ccls), necessitating a kill.

moreover, this might be an unpopular opinion, but mixing any regex based tool (ripgrep/... come to mind) with language-server f.e. because the language server does not really find what you are looking for, or does not do that fast enough, are major points against it. if you already have language-server running, regex based tools should not be required at all.

i don't really understand the reason for sql'ization of code searches at all. it is not a 'natural' interface. typical usage is to see 'who calls this function', 'where is the definition at' of this function etc. etc.

Post reply on HN