Check out https://NerdyData.com :-)
Show HN: Search code in GitHub repos using regular expressions
71–80 of 165 posts
Re: Show HN: Search code in GitHub repos using regular expressions
#72Earlier quoted context omitted.
Ripgrep is based on RE2 and supports backrefs. Wonder why they didn't use that.
Not quite. ripgrep uses Rust's regex engine, not RE2. Rust's regex engine is descended from RE2, but there is no code sharing. Rust's regex engine does not support backreferences. RE2 does not either. ripgrep does however have a -P/--pcre2 flag which causes it to use PCRE2 instead of Rust's regex engine. PCRE2 supports backreferences and other things, like look-around. (ripgrep also has an --auto-hybrid-regex flag, w…
Thanks for the clarification. As an aside, how difficult do you think it would be to compile ripgrep to wasm? In VS Code we use ripgrep for full-workspace search and Node's regex library for in-memory searches, but this leads to discrepancies and issues such as catastrophic backtracking in the in-memory search.
Re: Show HN: Search code in GitHub repos using regular expressions
#73A tangent, my biggest gripe with GitHub code search (within a repo) off the top of my head is the inability to blacklist directories or only search whitelisted directories. Often times I want to look up the implementation of a function, and bam, three pages of results from tests.
I'm glad I'm not the only one. It's very common that I'll be searching for a keyword that only appears in the actual code a handful of times but hundreds of times in tests. GitHub's search is practically useless in those cases.
I almost always just resort to cloning and searching with ripgrep, which can be annoying if I have no other reason to have the codebase on my machine or it's just a one-off.
Re: Show HN: Search code in GitHub repos using regular expressions
#74Earlier quoted context omitted.
Not quite. ripgrep uses Rust's regex engine, not RE2. Rust's regex engine is descended from RE2, but there is no code sharing. Rust's regex engine does not support backreferences. RE2 does not either. ripgrep does however have a -P/--pcre2 flag which causes it to use PCRE2 instead of Rust's regex engine. PCRE2 supports backreferences and other things, like look-around. (ripgrep also has an --auto-hybrid-regex flag, w…
Thanks for the clarification. As an aside, how difficult do you think it would be to compile ripgrep to wasm? In VS Code we use ripgrep for full-workspace search and Node's regex library for in-memory searches, but this leads to discrepancies and issues such as catastrophic backtracking in the in-memory search.
I've never tried to compile to WASM. It really depends on how much of the OS APIs need to be fixed. e.g., I don't think WASM supports memory maps as one example. In that case, ripgrep could be made to compile without support for memory maps with a bit of work. But that's an easy case. What other things does WASM not support? What about typical file/directory APIs? I don't think it does, or it least, it looks like Rust's standard library doesn't implement anything for them: https://github.com/rust-lang/rust/blob/master/src/libstd/sys...
At that point, it would be hopelessly difficult to build ripgrep. The right path then would be to build a new application that uses whatever of ripgrep's libraries make sense.
Popping up a level though, why would you want to compile to WASM? If you're using Node, then surely you can build an FFI bridge to Rust's regex library. At least at that point, you'd be using the same regex engine. I even maintain official C bindings for them: https://github.com/rust-lang/regex/tree/master/regex-capi
EDIT: Oh, and not sure if this is useful, but the regex crate itself should compile to WASM just fine. I know I've seen people run it in the browser before. If there's a problem here, then please file a bug!
Re: Show HN: Search code in GitHub repos using regular expressions
#75Amazin, why Microsoft hasn't built this for GitHub yet is beyond me. Can it grep on individual repos?
I built https://grephub.com for that. It doesn’t maintain an index so it’s not super snappy, but it’s good enough / better than you’d expect in many cases!
Re: Show HN: Search code in GitHub repos using regular expressions
#76Earlier quoted context omitted.
Thanks for the clarification. As an aside, how difficult do you think it would be to compile ripgrep to wasm? In VS Code we use ripgrep for full-workspace search and Node's regex library for in-memory searches, but this leads to discrepancies and issues such as catastrophic backtracking in the in-memory search.
I've never tried to compile to WASM. It really depends on how much of the OS APIs need to be fixed. e.g., I don't think WASM supports memory maps as one example. In that case, ripgrep could be made to compile without support for memory maps with a bit of work. But that's an easy case. What other things does WASM not support? What about typical file/directory APIs? I don't think it does, or it least, it looks like Rus…
Thanks for all the advice. We'd just be running it on single buffers so I agree it makes more sense to start from the rust regex library than ripgrep. We do however need to continue supporting backrefs and lookaround, so we'll need to add `--auto-hybrid-regex` functionality to fall back to either Node's engine or a webassembly PCRE2.
As for wasm vs FFI, it would ideally work in browser (Monaco), which makes wasm the best bet I believe.
Re: Show HN: Search code in GitHub repos using regular expressions
#77Earlier quoted context omitted.
I've never tried to compile to WASM. It really depends on how much of the OS APIs need to be fixed. e.g., I don't think WASM supports memory maps as one example. In that case, ripgrep could be made to compile without support for memory maps with a bit of work. But that's an easy case. What other things does WASM not support? What about typical file/directory APIs? I don't think it does, or it least, it looks like Rus…
Thanks for all the advice. We'd just be running it on single buffers so I agree it makes more sense to start from the rust regex library than ripgrep. We do however need to continue supporting backrefs and lookaround, so we'll need to add `--auto-hybrid-regex` functionality to fall back to either Node's engine or a webassembly PCRE2. As for wasm vs FFI, it would ideally work in browser (Monaco), which makes wasm the…
Ah yeah, for backrefs you'll need to find a way to use PCRE2. Not sure what the WASM story is there. But at that point, if your only problem with Node regexes is catastrophic backtracking, then you might as well just stick with Node. PCRE2 will have the same problem.
Re: Show HN: Search code in GitHub repos using regular expressions
#78[deleted]
Re: Show HN: Search code in GitHub repos using regular expressions
#79This is really cool! Awesome work. I assume you've seen https://sourcegraph.com/ as well? This to me seems much clearer and a bit more intuitive (though I've only spent a little time in sourcegraph). Really really cool. Does it also search code comments?
Re: Show HN: Search code in GitHub repos using regular expressions
#80Earlier quoted context omitted.
There must be something else or something wrong, because you indexed one of my small repo (~100 stars, ~20 forks, ~20Mb) and not the bigger ones (~500 stars, ~100/150 forks, ~150Mb)
Maybe he is limiting it to repositories of 50 MB or less, for example.
or possibly there's a "time decay" element where more recently "popular" repos are prioritized, not just based on absolute start/fork count