31–40 of 164 posts
Earlier quoted context omitted.
I've used both Code Search and Livegrep. No, Livegrep does not even come close to what Code Search can do. Sourcegraph is the closest thing I know of.
Is there like a summary of what's missing from public attempts and what makes it so much better?
Surprised that hound https://github.com/hound-search/hound isn't mentioned. I thought it was the leader of open source solutions in this space. I've been using Wikimedia's instance ( https://codesearch.wmcloud.org/search/ ) and have generally been pretty happy with what it provides.
Properly bounding search response sizes was one of the things I had to ensure Zoekt could do in its JSON APIs that I use in neogrok: https://github.com/sourcegraph/zoekt/pull/615
To me it doesn't make sense to be more scared of it than Elasticsearch...
I suppose using something like tree sitter to get a consistent abstract syntax tree to work with would be a good starting point. And then try building a custom analyzer (if using elasticsearch lingo) with that?
But a tokenizer seems like a good place to start, I think that's the right granularity for this kind of application. You'd want to do some chunking so that foo.bar doesn't find every foo and every bar, that sort of thing. Code search is, as the title says, a hard problem. But a language-aware token stream, the one you'd get from the lexer, is probably where one should start in building the database.
Earlier quoted context omitted.
Is there like a summary of what's missing from public attempts and what makes it so much better?
The short answer is context. The reason why Google's internal code search is so good, is it is tied into their build system. This means, when you search, you know exactly what files to consider. Without context, you are making an educated guess, with regards to what files to consider.
I don't understand their hand-waving of Zoekt. It was built exactly for this purpose, and is not a "new infrastructure commitment" any more than the other options. The server is a single binary, the indexer is also a single binary, can't get any simpler than that. To me it doesn't make sense to be more scared of it than Elasticsearch...
Earlier quoted context omitted.
If you ever leave you can use Livegrep, which was based on code-search work done at Google. I personally don't use it right now but it's great and will probably meet all your needs. [0] https://github.com/livegrep/livegrep
I've used both Code Search and Livegrep. No, Livegrep does not even come close to what Code Search can do. Sourcegraph is the closest thing I know of.
Nix should also have enough context so that for any project it can get the source code of all dependencies and (optionally) all build-time dependencies.
It indeed is hard, and a good code search platform makes life so much easier. If I ever leave Google, the internal code search is for sure going to be the thing I miss the most. It's so well integrated into how everything else works (blaze target finding, guice bindings etc), I can't imagine my life without it. I remember to appreciate it even more every time I use Github's search. Not that it's bad, it's just inhere…
If you ever leave you can use Livegrep, which was based on code-search work done at Google. I personally don't use it right now but it's great and will probably meet all your needs. [0] https://github.com/livegrep/livegrep
If I’ve learned anything from the fainting spells that I-work-at-X have over their internal tools on HN: no, whatever the public/OSS variant is always a mere shadow of the real thing.
I don't understand their hand-waving of Zoekt. It was built exactly for this purpose, and is not a "new infrastructure commitment" any more than the other options. The server is a single binary, the indexer is also a single binary, can't get any simpler than that. To me it doesn't make sense to be more scared of it than Elasticsearch...