Ripgrep – A new command line search tool

Post by burntsushi » Sat, Sep 24, 2016, 3:45 AM UTC

JoshTriplett wrote: post #138

ggreer wrote: post #63

I'm the author of ag. That was a really good comparison of the different code searching tools. The author did a great job of showing how each tool misbehaved or performed poorly in certain circumstances. He's also totally right about defaults mattering. It looks like ripgrep gets most of its speedup on ag by: 1. Only supporting DFA-able Rust regexes. I'd love to use a lighter-weight regex library in ag, but users are…

> 1. Only supporting DFA-able Rust regexes. I'd love to use a lighter-weight regex library in ag, but users are accustomed to full PCRE support. Switching would cause me to receive a lot of angry emails. Maybe I'll do it anyway. PCRE has some annoying limitations. (For example, it can only search up to 2GB at a time.) The standard trick here is to use the faster method for searches that it supports, and use the slowe…

Post by tonglil » Sat, Sep 24, 2016, 3:56 AM UTC

burntsushi wrote: post #146

ggreer wrote: post #139

Earlier quoted context omitted.

…and merged: https://github.com/ggreer/the_silver_searcher/pull/974 I'll tag a new release in a day or two. Also, it looks like the sift author is getting on the .ignore train: https://github.com/svent/sift/issues/78#issuecomment-2493352... This worked out pretty well. :)

Same! https://github.com/BurntSushi/ripgrep/pull/41 Agreed :-)

Post by burntsushi » Sat, Sep 24, 2016, 4:02 AM UTC

bodyfour wrote: post #154

burntsushi wrote: post #70

Earlier quoted context omitted.

One thing I did benchmark was the use of memory maps for single file search (cf. `subtitles_literal`). In that case, it saved time (small, but measurable ) to memory map the file than to incrementally read it. Memory maps were only slower in parallel search on large directories. Thankfully, ripgrep makes it easy to switch between memory maps and incremental reading. So I can just do this for you right now on the spot…

Interesting that user time went up as well.. not sure if that's significant. I guess it's not too surprising that mmap isn't much of a win these days for anything... SIMD can copy a memory page pretty fast these days. I just installed rg from homebrew and it's quite impressive... about 2.5x faster than ag on my macbook pro. Interestingly I get another 25% improvement by falling back to -j3 even though I'm on a quad-c…

Post by petre » Sat, Sep 24, 2016, 4:12 AM UTC

ggreer wrote: post #103

burntsushi wrote: post #66

Earlier quoted context omitted.

Thanks for the response! Some notes: 1. In my benchmarks, I do control for line numbers by either explicitly making it a variable (i.e., when you see `(lines)`) or by making all tools count lines to make the comparison fair. For the most part, this only tends to matter in the single-file benchmarks. 2. For memory maps, you might get very different results depending on your environment. For example, I enabled memory m…

In terms of core features, ripgrep is totally there. It searches fast . It ignores files pretty accurately. It outputs results in a pleasant and useful format. If a new user tries rg, they'll be very happy. My warning about the feature differences was meant to temper ag users' expectations. There are lots of little things that ag users are accustomed to that are either different or missing in ripgrep. Off the top of…

Post by [deleted] » Sat, Sep 24, 2016, 4:30 AM UTC

burntsushi wrote: post #135

ggreer wrote: post #131

Earlier quoted context omitted.

I also vote for "do what gitignore does". My plan is to add support for the new file name, deprecate .agignore, and update docs everywhere. But it'd be a while before I removed .agignore completely. I really like .ignore, and I like it because it's generic. The information I want it to convey is: > Dear programs, > If you are traversing this directory, please ignore these things. Of course, some programs could still…

Let's do it. https://github.com/BurntSushi/ripgrep/issues/40