Code search is hard • Hacker News Board

Re: Code search is hard

#161

Post by hiAndrewQuinn » Sat, Apr 13, 2024, 5:40 AM UTC

mcintyre1994 wrote: post #124

hiAndrewQuinn wrote: post #110

Basic code searching skills seems like something new developers are never explicitly taught, but which is an absolutely crucial skill to build early on. I guess the knowledge progression I would recommend would look something kind this: - Learning about Ctrl+F, which works basically everywhere. - Transitioning to ripgrep https://github.com/BurntSushi/ripgrep - I wouldn't even call this optional, it's truly an incredi…

I’d also point out that VSCode uses ripgrep for its search feature which is a great starting point.

It does! And I only recently learned this, and it explains why I've always found the VS Code search and replace across all files to be tremendously useful.

Re: Code search is hard

#162

Post by ivanovm » Sun, Apr 14, 2024, 5:24 PM UTC

erikars wrote: post #156

ivanovm wrote: post #151

Earlier quoted context omitted.

My, and my friends experiences with SCIP indexers built by Sourcegraph have been less than stellar. They are buggy and sparsely maintained

Hello! I am Head of Engineering at Sourcegraph. I'd love to get feedback on which SCIP indexers you've had issues with, and, if you have the time, feedback on what sort of problems you've had with them. Thank you so much!

Hey guys, it's been over two months since I've been in the weeds with SCIP so I'm not going to be able to write very detailed issues, most of my experiences were with scip python and some in typescript.

1. roles incorrectly assigned to symbol occurences

2. symbols missing - this is a big one. I've seen many instances of symbols being included in "relationships" array that were not included in "symbols" array for the document, and vice versa. Plus "definition" occurrences have been inconsistent/confusing - only some symbols have those, and they don't always match where the thing is actually defined (file/position), and sometimes a definition occurrence has no counterpart in symbols array

3. the treatment of external packages have been inconsistent, they sometimes get picked up as internal definitions and sometimes not

I think SCIP is a great idea and I'd explore using it again if it got better. But I see that there are issues staying in the backlog for 6+ months which makes it seem from the outside like Sourcegraph is not prioritizing further development of scip

Re: Code search is hard

#163

Post by mmanela » Tue, Apr 16, 2024, 2:57 PM UTC

ivanovm wrote: post #162

erikars wrote: post #156

Earlier quoted context omitted.

Hello! I am Head of Engineering at Sourcegraph. I'd love to get feedback on which SCIP indexers you've had issues with, and, if you have the time, feedback on what sort of problems you've had with them. Thank you so much!

Hey guys, it's been over two months since I've been in the weeds with SCIP so I'm not going to be able to write very detailed issues, most of my experiences were with scip python and some in typescript. 1. roles incorrectly assigned to symbol occurences 2. symbols missing - this is a big one. I've seen many instances of symbols being included in "relationships" array that were not included in "symbols" array for the…

Thanks for the details. I appreciate you taking the time to give the feedback. We will use this to help to improve SCIP and the indexers.

Re: Code search is hard

#164

Post by johnthescott » Sun, Apr 21, 2024, 2:37 AM UTC

philippemnoel wrote: post #147

johnthescott wrote: post #87

the rum index has worked well for us on roughly 1TB of pdfs. written by postgrespro, same folks who wrote core text search and json indexing. not sure why rum not in core. we have no problems. https://github.com/postgrespro/rum

RUM is good, but it lacks some of the more complex features like language tokenizers, etc. that a full search engine library like Lucene/Tantivy (and ParadeDB in Postgres) offer

could you elaborate?

i was under the impression the text search config and dictionaries did support such remappings. we only query in english or literal, so my knowledge is limited.