Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Code search is hard

blog.val.town

Post reply on HN
Print view View original

151–160 of 164 posts

Re: Code search is hard

#151

Post by ivanovm »

mechanicker wrote: post #136
sqs wrote: post #15

I'm at Sourcegraph (mentioned in the blog post). We obviously have to deal with massive scale, but for anyone starting out adding code search to their product, I'd recommend not starting with an index and just doing on-the-fly searching until that does not scale. It actually will scale well for longer than you think if you just need to find the first N matches (because that result buffer can be filled without needing…

I hope this and SCIP becomes a standard and we have more programming languages emitting symbols in SCIP format.

My, and my friends experiences with SCIP indexers built by Sourcegraph have been less than stellar. They are buggy and sparsely maintained

Re: Code search is hard

#152

Post by __float »

mdaniel wrote: post #140
mechanicker wrote: post #136

Earlier quoted context omitted.

I hope this and SCIP becomes a standard and we have more programming languages emitting symbols in SCIP format.

I thought SCIP got promoted into https://lsif.dev/ but chasing the https://github.com/sourcegraph/lsif-java link resolves to https://github.com/sourcegraph/scip-java so maybe I had the evolution relationship backward. Anyway, I'm thankful at least that code is still Apache 2 https://github.com/topics/lsif may interest this audience, too, since the scip topic tag seems to clash with something else Also, I learned last…

SCIP is an evolution of LSIF, basically: https://sourcegraph.com/blog/announcing-scip

Re: Code search is hard

#153

Post by dmoy »

isker wrote: post #46
riku_iki wrote: post #35

Earlier quoted context omitted.

How exactly integration with build system helps Google? Maybe you could give specific example?..

Try clicking around https://source.chromium.org/chromium/chromium/src , which is built with Kythe (I believe, or perhaps it's using something internal to Google that Kythe is the open source version of). By hooking into C++ compilation, Kythe is giving you things like _macro-aware_ navigation. Instead of trying to process raw source text off to the side, it's using the same data the compiler used to compile the code…

Yes the clicking around on semantic links on source.chromoum.org is served off of an index built by the Kythe team at Google.

The internal Kythe has some interesting bits (mostly around scaling) that aren't open sourced, but it's probably doable to run something on chromium scale without too much of that.

The grep/search box up top is a different index, maintained by a different team.

Re: Code search is hard

#154

Post by hanwenn »

Hi,

I wrote zoekt. From what I understand valtown does, I would try to use brute force first (ie. something equivalent to ripgrep). Once that starts breaking down, you could use last-updated-timestamps to reduce the brute force:

* make a trigram index using Zoekt or Hound for JS snippets older than X * do brute force on snippets newer than X * advance X as you're indexing newer data.

If the snippets are small, you can probably use a (trigram => snippets) index for space savings relative to a (trigram => offset) index.

Re: Code search is hard

#155

Post by pomdtr »

pomdtr wrote: post #73

Hey! I'm a val.town fanboy and I immediately thought about a workaround while reading the blog post: What if I dumped every publics vals in Github, in order to be able to user their (awesome) search ? So here is my own "Val Town Search": https://val-town-search.pomdtr.me And here is the repo containing all vals, updated hourly thanks to a github action: https://github.com/pomdtr/val-town-mirror

I ended up writing my own article: https://code-search-is-easy.pomdtr.me/

Re: Code search is hard

#156

Post by erikars »

ivanovm wrote: post #151
mechanicker wrote: post #136

Earlier quoted context omitted.

I hope this and SCIP becomes a standard and we have more programming languages emitting symbols in SCIP format.

My, and my friends experiences with SCIP indexers built by Sourcegraph have been less than stellar. They are buggy and sparsely maintained

Hello! I am Head of Engineering at Sourcegraph. I'd love to get feedback on which SCIP indexers you've had issues with, and, if you have the time, feedback on what sort of problems you've had with them. Thank you so much!

Re: Code search is hard

#157

Post by yencabulator »

birktj wrote: post #37
init wrote: post #18

Earlier quoted context omitted.

I've used both Code Search and Livegrep. No, Livegrep does not even come close to what Code Search can do. Sourcegraph is the closest thing I know of.

I see most replies here ar mentioning the the build integration is what is mainly missing in the public tools. I wonder if nix and nixpkgs could be used here? Nix is a language agnostic build-system and with nixpkgs it has a build instructions for a massive amount of packages. Artifacts for all packages are also available via hydra. Nix should also have enough context so that for any project it can get the source cod…

Nix builds suck for development because there is no incrementality there. Any source file changes in any way, and your typical nix flake will rebuild the project from scratch. At best, you get to reuse builds of dependencies.

Re: Code search is hard

#158

Post by mmanela »

ivanovm wrote: post #151
mechanicker wrote: post #136

Earlier quoted context omitted.

I hope this and SCIP becomes a standard and we have more programming languages emitting symbols in SCIP format.

My, and my friends experiences with SCIP indexers built by Sourcegraph have been less than stellar. They are buggy and sparsely maintained

I work at Sourcegraph and would love to learn more.

1. Which SCIP indexers did you having issues with?

2. What issues did you hit (can you share details or link to GitHub issues filed?)

Thanks!

Re: Code search is hard

#159

Post by scubbo »

tripdout wrote: post #111
scubbo wrote: post #49

Earlier quoted context omitted.

I suspect you're being sarcastic - but can confirm that being nearly two years out of Amazon, I still miss its in-house CD system nearly every day. I've actively looked around for OSS replacements and very few come anywhere close. (I would be _delighted_ for someone to "Umm actually" me by providing a great product!)

Is it true that teams don't do branches in source control at all? Just publishing a CR?

Yep! Well, a branch is created under-the-hood, and you can still check it out locally if you want to - but the standard code contribution workflow doesn't require you to think or work in those terms at all.

Re: Code search is hard

#160

Post by erikars »

erikars wrote: post #156
ivanovm wrote: post #151

Earlier quoted context omitted.

My, and my friends experiences with SCIP indexers built by Sourcegraph have been less than stellar. They are buggy and sparsely maintained

Hello! I am Head of Engineering at Sourcegraph. I'd love to get feedback on which SCIP indexers you've had issues with, and, if you have the time, feedback on what sort of problems you've had with them. Thank you so much!

[dead]
Post reply on HN