Towards Natural Language Semantic Code Search at GitHub

Post by VirenM » Wed, Sep 19, 2018, 3:37 AM UTC

dgreensp wrote: post #14

It is a real cultural problem how engineers get more excited about machine learning than basic usability. GitHub search can't even search for a literal string, let alone a regex. It can't search a subdirectory. Ranking is indistinguishable from random. It's been this way for years. How about building an actual, usable, basic code search and then getting all fancy with your machine learning? I almost built my own "onl…

Post by hv42 » Wed, Sep 19, 2018, 6:21 AM UTC

hiccuphippo wrote: post #20

samlambert wrote: post #9

Earlier quoted context omitted.

We are very aware of the problem. I think you are going to really love what we are working on.

I really hope you are right and have your priorities straight when it comes to search. I'd love for a way to search for usages of a class::method, or for strings that contain the text "hello" or for variables named foo. And if you integrate that into the code itself, Ctrl+click a class method to find all usages, maybe even usages in other repositories, so I can see how other people use a certain library. And of cours…

Post by bryanrasmussen » Wed, Sep 19, 2018, 6:50 AM UTC

nerdponx wrote: post #34

boyter wrote: post #30

Earlier quoted context omitted.

People use grep on their local code repository, which is generally less than 2 gigabytes of source. A tool like ripgrep can process that in under a second on any modern machine with a warm disk cache. Its when you get to hundreds of repositories or 10's of gigabytes of code that local tools cannot run fast enough. They are not designed for this use-case, and usually rely on the files being searched hitting the disk c…

Nobody is asking for cross-repo search. Literally just let us run "git grep" on Github.

Post by sqs » Wed, Sep 19, 2018, 7:30 AM UTC

DannyBee wrote: post #42

I saw this created in another thread and it seems to accurately sum up the comments here: https://imgflip.com/i/2i90x2

Post by amelius » Wed, Sep 19, 2018, 8:35 AM UTC

derefr wrote: post #36

nerdponx wrote: post #34

Earlier quoted context omitted.

Nobody is asking for cross-repo search. Literally just let us run "git grep" on Github.

On your local machine, you can get away with a linear scan (as "git grep" does), because nothing else is contending for time with your search. "git grep" is actually very costly in terms of CPU and disk IO, but you don't notice, because you're only running one "git grep" at once, and so nothing is contending for those resources. On GitHub, tens of thousands of searches will be happening at once on the same search clu…