Looks like every tool has its upsides and downsides. This one lacks full PCRE syntax support. Does one have to install Rust to use it?
No. binaries: http://blog.burntsushi.net/ripgrep/#installation
Ripgrep – A new command line search tool
71–80 of 219 posts
Re: Ripgrep – A new command line search tool
#72Nice! Lightgrep[1] uses libicu et al to look up code points for a user-specified encoding and encode them as bytes, then just searches for the bytes. Since ripgrep is presumably looking just for bytes, too, and compiling UTF-8 multibyte code points to a sequence of bytes, perhaps you can do likewise with ICU and support other encodings. ICU is a bear to build against when cross-compiling, but it knows hundreds of enc…
I hadn't heard of liblightgrep, nice. It's on my short list for looking more closely.
I doubt I'd ever be comfortable with Rust's regex engine growing a dependency on libicu, but it's still worth understanding your implementation. Some questions, if you don't mind. The big one is: does your regex engine use finite automata, and does it put the text decoding into the automaton itself? For example, when you compile the `.` regex, you end up with an automaton that inlines UTF-8 decoding itself. It looks like this: https://gist.github.com/anonymous/8fbe170bfcca5d7475b59299fa...
Does you regex library do that for each type of encoding? Or is there a transcoding step?
Re: Ripgrep – A new command line search tool
#731. Ag have nice editor integration. I would miss emacs helm-projectile-ag
2. Pcre is good regexp flavor to master. It is have good balance of speed, power and popularity. In addition to Ag, there are accessible libraries in many languages, including python.
I think it would be good if everyone settled on Pcre, rather than each language thinking they will do regexps better.
Re: Ripgrep – A new command line search tool
#74Looks like every tool has its upsides and downsides. This one lacks full PCRE syntax support. Does one have to install Rust to use it?
No, Rust tools don't need a Rust environment installed.
Re: Ripgrep – A new command line search tool
#75Earlier quoted context omitted.
I agree, and I'm already using both ripgrep and rust-parallel ( https://github.com/mmstick/parallel , a gnu-parallel replacement which should probably get another name ). I am really happy to see Rust apps actually been written -- Rust programmers seem to be actually trying to replace the existing code lying around, instead of just insulting it and telling us how much better the world would be if we used their langua…
Well Burntsushi, the author is quite famous and level headed programmer who has written in both Go and Rust. He is exception to what you describe of Rust programmers who are still not much changed.
Actually, I'm no exception at all. The Rust community (and the Go community) is full of level headed exceptional people. I love working with them!
Re: Ripgrep – A new command line search tool
#76Earlier quoted context omitted.
Is there anything about ripgrep that makes its performance uniquely fast? That is, is it not possible to achieve the same speed and correctness of results using, say, C or C++? If neither condition holds, then the use of Rust is almost purely incidental. It isn't even interesting, from that perspective. Far more interesting would be a measure of how safe the Rust code is versus equivalent-results implementations in o…
It seems to be faster than grep primarily by having fewer features (it doesn't even attempt to support all of POSIX regular expressions, for one).
To be clear, I do not believe ripgrep is faster than grep because it has fewer features. At least, I've yet to see any evidence to the contrary.
I did try to explain why I think ripgrep is faster in all kinds of gory details though. :-)
Re: Ripgrep – A new command line search tool
#77I'm never sure whether or not I should adopt these fancy new command line tools that come out. I get them on my muscle memory and then all of a sudden I ssh into a machine that doesn't have any of these and I'm screwed...
I have to switch back and forth between ack and grep all the time. Sometimes I use ack, sometimes I use grep. I wrote ack, but I've never stopped using grep, and ack has never been intended as a replacement for grep.
For most of the common ack flags, they are the same as grep. -i, -l, -v, -w, -A, -B, -C, etc. That was intentional, to minimize that whiplash.
One other suggestion is that you drop ack into your ~/bin directory that you sync between machines. It's a single Perl file, and that portability is a feature. As long as your machine has a Perl on it, you can use ack.
Re: Ripgrep – A new command line search tool
#78Earlier quoted context omitted.
I agree, and I'm already using both ripgrep and rust-parallel ( https://github.com/mmstick/parallel , a gnu-parallel replacement which should probably get another name ). I am really happy to see Rust apps actually been written -- Rust programmers seem to be actually trying to replace the existing code lying around, instead of just insulting it and telling us how much better the world would be if we used their langua…
This would have been a lot more exciting if it were designed to actually be even slightly compatible with grep (or maybe had some core in there that was, while leaving the other UI parts he wants to change on top) and were then approaching GNU and saying "hey, this is something I've been working on: would you consider making grep the first standalone tool to move to Rust, and what would it take from me to make this h…
> (or maybe had some core in there that was, while leaving the other UI parts he wants to change on top)
But I did! Have you looked at the dependency list of ripgrep? It's utterly filled with tons of tools that you can pick out and use in other projects for any purpose you like. I didn't mention this in the blog post because there's already too much there, but sure, here they are:
memchr - Fast single byte search: http://burntsushi.net/rustdoc/memchr/
walkdir - Recursive directory iterator: http://burntsushi.net/rustdoc/walkdir/
utf8-ranges - Generate utf8 automata: http://burntsushi.net/rustdoc/utf8_ranges/
regex-syntax - A regex parser (including literal extraction helpers): https://doc.rust-lang.org/regex/regex_syntax/index.html
regex - The regex engine itself: https://doc.rust-lang.org/regex/regex/index.html
grep - Line-by-line search (as a library). This is where all of the inner literal optimizations happen, for example. http://burntsushi.net/rustdoc/grep/
And this is only the stuff that I did. This doesn't count all the other wonderful stuff I used that other folks built!
And sure, I could do better. There's more stuff I could move into the `grep` crate, but this is only the beginning, not the end.
> "hey, this is something I've been working on: would you consider making grep the first standalone tool to move to Rust, and what would it take from me to make this happen?"
But I didn't want to do that. I don't understand why that's a problem. Doing this requires being POSIX interface compatible, and that's not a hole I care to dig. I don't mean any disrespect, it's just not what I want to spend my time doing.
> as opposed to writing an article about how "I am smarter than the GNU grep people for these reasons and have built a tool named after how my tool is going to kill their tool" which almost seems to go out of its way to set up an air of competition rather than collaboration
I'm really sorry if I came across that way. It was of course not my intention. My intention was to write about what I had learned, which seems like a pretty fundamental component of collaboration. I'm sure five years from now, when I've moved on to other problems, someone will come along and beat ripgrep, and I can only hope that they write about it. :-)
I mean, if it weren't for the innumerable people who wrote about their experience with this kind of stuff, I never would have been able to get here. It only makes sense for me to write if I think I have something valuable to share.
> (I mean: seriously... "ripgrep"?! This developer is clearly going out of their way to be combative.
I'm not. "rip" was supposed to mean "rip through text." I'm sorry it came off as combative. I wasn't even aware of the "rest in peace" interpretation until someone else pointed it out.
Honestly... I was trying hard to find a way to justify the binary name `rg` because I liked that `r` could stand for Rust. But `rustgrep` seemed a bit too in-your-face, so I started searching for small relevant words starting with `r`. That's it.
Re: Ripgrep – A new command line search tool
#79Earlier quoted context omitted.
This would have been a lot more exciting if it were designed to actually be even slightly compatible with grep (or maybe had some core in there that was, while leaving the other UI parts he wants to change on top) and were then approaching GNU and saying "hey, this is something I've been working on: would you consider making grep the first standalone tool to move to Rust, and what would it take from me to make this h…
Is there anything about ripgrep that makes its performance uniquely fast? That is, is it not possible to achieve the same speed and correctness of results using, say, C or C++? If neither condition holds, then the use of Rust is almost purely incidental. It isn't even interesting, from that perspective. Far more interesting would be a measure of how safe the Rust code is versus equivalent-results implementations in o…
I can't think of any particular piece that requires Rust specifically for performance, no. The real beauty of Rust is that I was able to achieve this performance in the first place, all with very very little use of unsafe (even in the regex engine itself).
But of course, as we all know, performance isn't everything. Even ripgrep itself accepts a performance penalty in its default mode of operation, in order to improve the relevance of results shown.
Re: Ripgrep – A new command line search tool
#80I'm the author of ag. That was a really good comparison of the different code searching tools. The author did a great job of showing how each tool misbehaved or performed poorly in certain circumstances. He's also totally right about defaults mattering. It looks like ripgrep gets most of its speedup on ag by: 1. Only supporting DFA-able Rust regexes. I'd love to use a lighter-weight regex library in ag, but users are…
Thanks for the response! Some notes: 1. In my benchmarks, I do control for line numbers by either explicitly making it a variable (i.e., when you see `(lines)`) or by making all tools count lines to make the comparison fair. For the most part, this only tends to matter in the single-file benchmarks. 2. For memory maps, you might get very different results depending on your environment. For example, I enabled memory m…
`ag --pager` is the most important one (I use alias ag='ag --pager "less -FRX"')