Live data from Hacker News

Ripgrep is faster than grep, ag, Git grep, ucg, pt, sift (2016)

blog.burntsushi.net

71–80 of 198 posts

Re: Ripgrep is faster than grep, ag, Git grep, ucg, pt, sift (2016)

#71
post #38

What are the reasons for grep not being replaced/improved? This topic seems a bit old by now.

I guess it is because after decades of use, grep has probably been fixed to handle lots of user cases that the new tools don´t handle because they haven´t found them yet.

Re: Ripgrep is faster than grep, ag, Git grep, ucg, pt, sift (2016)

#72

Earlier quoted context omitted.

And also the SIMD in aho-corasick, which is used whenever a small number of literals are searched for. For example, `foo|bar` or `(?i)foo`. https://github.com/BurntSushi/aho-corasick/blob/f227162f7c56... But no `_mm256_sad_epu8`. What an oddly specific question..?

What is really oddly specific is this instruction :p but I believe it is a common trick to quickly scan for short string matches. It computes `sum(|x[i] - y[i]|)` for consecutive `i` at different offsets, so it should be zero at substring matches. For context: https://epubs.siam.org/doi/pdf/10.1137/1.9781611972931.10 I was slightly mistaken, the instruction of interest is _mm256_mpsadbw_epu8

Oh I see. Yes, that's what is commonly used in academic publications. But I've yet to see it used in the wild.

I mentioned exactly that paper (I believe) in my write-up on Teddy: https://github.com/BurntSushi/aho-corasick/tree/master/src/p...

Re: Ripgrep is faster than grep, ag, Git grep, ucg, pt, sift (2016)

#74
post #67
post #3

It's fast indeed. And I can't help keeping promoting the combination with fzf :) For those who want to try it out, this is a Powershell function but the same principle applies in any shell. Does ripgrep then puts fuzzy searching in the resulting files+text on top while showing context in bat: function frg { $result = rg --ignore-case --color=always --line-number --no-heading @Args | fzf --ansi ` --color 'hl:-1:underl…

Oh. Thanks for the tip. This might make me finally embrace powershell. I’ve been using WSL+zsh+fzf as a Windows CLI for continuity with day job Mac tools, but git CLI performance is only usable inside the WSL file system.

You can also add a small script to your WSL under `/usr/local/bin/git`:

  GIT_WINDOWS="/mnt/c/Program Files/Git/bin/git.exe"
  GIT_LINUX="/usr/bin/git"
  
  case "$(pwd -P)" in
  /mnt/?/*)
    case "$@" in
    # Needed to fix prompt, but it breaks things like paging, colours, etc
    rev-parse*)
      # running linux git for rev-parse seems faster, even without translating paths
      exec "$GIT_LINUX" "$@"
      ;;
    *)
      exec "$GIT_WINDOWS" -c color.ui=always "$@"
      ;;
    esac
    ;;
  *)
    exec "$GIT_LINUX" "$@"
    ;;
  esac

This allows you to use `git` in your WSL shell but it'll pick whichever executable is suitable for the filesystem that the repo is in :)

Re: Ripgrep is faster than grep, ag, Git grep, ucg, pt, sift (2016)

#75
post #71
post #38

What are the reasons for grep not being replaced/improved? This topic seems a bit old by now.

I guess it is because after decades of use, grep has probably been fixed to handle lots of user cases that the new tools don´t handle because they haven´t found them yet.

Author of ripgrep here.

Like automatic encoding detection and transparently searching UTF-16?

Or simple ways for composing character classes, e.g., `[\pL&&\p{Greek}]` for all codepoints in the Greek script that are letters. Another favorite of mine is `\P{ascii}`, which will search for any codepoint that isn't in the ASCII subset.

Or more sophisticated filtering features that let you automatically respect things like gitignore rules.

Those are all things that ripgrep does that grep does not. So I do not favor this explanation personally.

ripgrep has just about all of the functionality that GNU grep does. I would say the two biggest missing pieces at this point are:

* POSIX locale support. (But this might be a feature[1].)

* Support for "basic" regexes or some equivalent that flips the escaping rules around. i.e., You need to write `\+` to match 1 or more things, where as `+` will just match `+ literally.

Otherwise, ripgrep has unfortunately grown just about as many flags as GNU grep.

[1]: https://github.com/mpv-player/mpv/commit/1e70e82baa9193f6f02...

Re: Ripgrep is faster than grep, ag, Git grep, ucg, pt, sift (2016)

#76

Earlier quoted context omitted.

And also the SIMD in aho-corasick, which is used whenever a small number of literals are searched for. For example, `foo|bar` or `(?i)foo`. https://github.com/BurntSushi/aho-corasick/blob/f227162f7c56... But no `_mm256_sad_epu8`. What an oddly specific question..?

What is really oddly specific is this instruction :p but I believe it is a common trick to quickly scan for short string matches. It computes `sum(|x[i] - y[i]|)` for consecutive `i` at different offsets, so it should be zero at substring matches. For context: https://epubs.siam.org/doi/pdf/10.1137/1.9781611972931.10 I was slightly mistaken, the instruction of interest is _mm256_mpsadbw_epu8

I think Wojciech Muła, who devised the original SIMD-oriented Rabin Karp algorithm, also did measure MPSADBW approaches and found that it is not a good fit for general string-in-string searches [1]. Maybe not today though.

[1] http://0x80.pl/articles/simd-strfind.html#id7

Re: Ripgrep is faster than grep, ag, Git grep, ucg, pt, sift (2016)

#77
post #60
post #30

Earlier quoted context omitted.

So I was casually searching for "ugrep vs ripgrep" articles, when I stumbled upon a couple reddit posts where apparently the authors of ugrep and ripgrep seemed to have a multi-year feud on reddit, eg. https://www.reddit.com/r/programming/comments/120wqvr/ripgre... So weird. I mean, it's just about some open source tool, right? :-/

This is so weird, even ripgrep's author is actively seeking conflict in ugrep's new release posts. Not a good colour on both of them.

At least how I read it the linked post was ugrep's author seeking conflict, not the ripgrep's.

Re: Ripgrep is faster than grep, ag, Git grep, ucg, pt, sift (2016)

#79
post #23

well... it is not faster than qgrep :) even though the way both work - differs greatly, and even though qgrep is based on re2 - the speed comes from the presence of index. but then I wonder why people forget the qgrep option, since with large file stores it makes much more sense to use qgrep AND indices, rather than always go through all the files. this above all true UNLESS you need multi-line matches with UTF8, whe…

Author of ripgrep here. Yes, qgrep uses indexing, which will always give it a leg up over other tools that don't use indexing. But of course, now you need to setup and maintain an index. The UX isn't quite as simple as "just run a search." But there isn't much of a mystery here. Someone might neglect to use qgrep for exactly the same reason that "grep is fast enough for me" might prevent someone from using ripgrep. A…

Does `build.rs` build the project? One of my favorite (Big Corp) code-bases just had a single C file (build.c) that did all the dependency tracking, like, Make, but in some nicely written (easy to understand) C code. The C file started with a shebang: a self-building-and-executing line, so we'd do this:

``` ./build.c ```

... and then magic happened.

Re: Ripgrep is faster than grep, ag, Git grep, ucg, pt, sift (2016)

#80

Earlier quoted context omitted.

Aeropress is a direct upgrade from french press and uses way less coffee

I hadn't heard of this before. Thanks!

Add one or a few drops of water to your roasted coffee beans with your hand and shake well after weighing it out to stop the grinds from sticking to the walls of your grinder from static.
Post reply on HN