Show HN: Search code in GitHub repos using regular expressions • Hacker News Board

Re: Show HN: Search code in GitHub repos using regular expressions

#41

Post by dang » Sun, Feb 23, 2020, 7:36 PM UTC

sonicxxg wrote: post #19

Typical HN comment. I could swear, if someone came here to share a breakthrough cure to HIV, some random nobody would still find a way to say something like "Hey, how come you haven't tested it against a population of female midgets suffering from cancer?".

There's no need for personal attack. We ban accounts that do that, so please don't.

Cherry-picking one post from a statistical cloud and calling it typical is dodgy. Even the distribution in this thread doesn't match your description. Actually, even the comment you're picking on doesn't match your description.

We detached this subthread from https://news.ycombinator.com/item?id=22397156.

Re: Show HN: Search code in GitHub repos using regular expressions

#42

Post by [deleted] » Sun, Feb 23, 2020, 7:41 PM UTC

sonicxxg wrote: post #19

Typical HN comment. I could swear, if someone came here to share a breakthrough cure to HIV, some random nobody would still find a way to say something like "Hey, how come you haven't tested it against a population of female midgets suffering from cancer?".

[deleted]

Re: Show HN: Search code in GitHub repos using regular expressions

#43

Post by dang » Sun, Feb 23, 2020, 7:47 PM UTC

I still miss Google Code Search, which was a great way to find examples of anything I wanted to learn about in programming and usually answered my questions better than anything else, including Stack Overflow. Has it really been 8 years... https://news.ycombinator.com/item?id=3112029

If this tool can fill that hole in my world, I'll be stoked. I've bookmarked it.

Re: Show HN: Search code in GitHub repos using regular expressions

#44

Post by rattray » Sun, Feb 23, 2020, 7:52 PM UTC

danfox wrote: post #11

simonw wrote: post #2

Impressive! Really fast, full featured code search across a huge corpus. 1. How did you build the index? Did you use a GitHub dump of some sort? How often do you refresh it? 2. Is it Elasticsearch or similar or a completely custom engine? 3. What kind of RAM/CPU are you using to power it? 4. Any plans to open source the code or commercialize the technology? I could absolutely imagine paying for a private code search…

Thanks! It's built on top of Solr. It fetches the repos from GitHub - it should pick up any updates to repos within a few days. It's running on a couple servers with 20 cores each, which is not really enough for the traffic it's getting right now.

Have you seen livegrep?

Blazing fast multi-repo regex code search. May be more expensive to run in prod, not sure.

Re: Show HN: Search code in GitHub repos using regular expressions

#45

Post by neonate » Sun, Feb 23, 2020, 7:52 PM UTC

thanatos_dem wrote: post #32

Next post from danfox - “how to get 3 job offers in 3 hours”. Already has been publicly contacted by: - GitHub CTO - SerpApi CEO - SourceGraph CEO Search is hot right now!

Also by the co-creator of Django: https://news.ycombinator.com/item?id=22397023

Re: Show HN: Search code in GitHub repos using regular expressions

#46

Post by jakear » Sun, Feb 23, 2020, 8:07 PM UTC

Any plans to include backrefs? I'd like to see how many examples of /(\w+) && \1\./ are out there in .js/.ts compared to /(\w+)\?\./

Re: Show HN: Search code in GitHub repos using regular expressions

#47

Post by [deleted] » Sun, Feb 23, 2020, 8:13 PM UTC

hoorayimhelping wrote: post #15

to grep specific repos locally, I use a tool called Hound, https://github.com/hound-search/hound developed by a couple of engineers at Etsy while I was there, but never released officially.

[deleted]

Re: Show HN: Search code in GitHub repos using regular expressions

#48

Post by danielecook » Sun, Feb 23, 2020, 8:21 PM UTC

Wow. This is incredibly helpful. You can use it to see how someone may have used a function with named parameters:

  my_function(label=x, option_1=2)
  my_function.*option_1 # search

Re: Show HN: Search code in GitHub repos using regular expressions

#49

Post by sdesol » Sun, Feb 23, 2020, 8:25 PM UTC

thanatos_dem wrote: post #37

Existenceblinks wrote: post #33

Earlier quoted context omitted.

I'm surprised as well, think why big tech companies didn't have this awesome search already.

If this were to be offered by an actual company (a first party solution), there are some features that'd be expected that make the problem space a lot harder. Here's an "intro to search" article that's a good read, and I'll use it to highlight some of the things that'd be different in a first party solution - https://medium.com/startup-grind/what-every-software-enginee... (See the "Theory: the search problem" section…

For GitHub, I would have to imagine only being able to search public repos with regexp would be good enough. GitHub has many strategies, but the main one is, they want to maintain, if not, expand their open source mind share.

The more reasons you give people to go to GitHub, the better off they will be in the future. So I do agree with you that as a commercial solution, this may not be viable, but for GitHub's public repos, this can turn into a very positive thing.

Re: Show HN: Search code in GitHub repos using regular expressions

#50

Post by leni536 » Sun, Feb 23, 2020, 8:28 PM UTC

big_chungus wrote: post #35

funklute wrote: post #22

Earlier quoted context omitted.

Why would you want to use this tool to grep individual repos? If you know the repo you're interested in, you can just clone it and then grep it locally...?

Some things can take a while to clone. On the top end, repos like blink, webkit, and gecko can take half an hour or more.

Even with --depth=1?