Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ripgrep 15.0

github.com

Post reply on HN
Print view View original

41–50 of 119 posts

Re: Ripgrep 15.0

#41

Post by oever »

This week I wrote a small bash function that run ripgrep only on the files that are tracked by git:

    rgg() {
        readarray -d '' -t FILES
It speeds up a lot on directories with many binary files and committed dot files. To search the dot files, -uu is needed, but that also tells ripgrep to search the binary files.

On repositories with hundreds of files, the git ls-files overhead a bit large.

Re: Ripgrep 15.0

#42

Post by WJW »

dotancohen wrote: post #20
vessenes wrote: post #10

rg is a first for me in that it's a CLI tool that an LLM taught me about -- it's a go-to tool for Claude and codex, and since I got most of my bash skills pre-dotcom-one-boom I'm historically just a grep user. Anyway I'm trying to retrain the fingers these days, rg is super cool.

Through I use rg to initiate searches, my muscle memory keeps using grep after pipes.

Huh I hadn't even realized I did that. I think grep has the "filter in pipe" spot in my head while rg has the "search recursively in all files" spot.

Re: Ripgrep 15.0

#43

Post by oever »

oever wrote: post #41

This week I wrote a small bash function that run ripgrep only on the files that are tracked by git: rgg() { readarray -d '' -t FILES It speeds up a lot on directories with many binary files and committed dot files. To search the dot files, -uu is needed, but that also tells ripgrep to search the binary files. On repositories with hundreds of files, the git ls-files overhead a bit large.

After writing this comment, I read the man page again and found the -. flag which can be used instead of -uu.

Searching in hidden files tracked by git would be great but the overhead of querying git to list all tracked files is probably significant even in Rust.

Re: Ripgrep 15.0

#44

Post by woodruffw »

oever wrote: post #41

This week I wrote a small bash function that run ripgrep only on the files that are tracked by git: rgg() { readarray -d '' -t FILES It speeds up a lot on directories with many binary files and committed dot files. To search the dot files, -uu is needed, but that also tells ripgrep to search the binary files. On repositories with hundreds of files, the git ls-files overhead a bit large.

Maybe I’m missing something, but doesn’t ripgrep ignore untracked files in git by default already?

Re: Ripgrep 15.0

#45

Post by burntsushi »

oever wrote: post #41

This week I wrote a small bash function that run ripgrep only on the files that are tracked by git: rgg() { readarray -d '' -t FILES It speeds up a lot on directories with many binary files and committed dot files. To search the dot files, -uu is needed, but that also tells ripgrep to search the binary files. On repositories with hundreds of files, the git ls-files overhead a bit large.

Can you provide a concrete example where that's faster? ripgrep should generally already be approximating `git ls-files` by respecting gitignore.

Also, `-uu` tells ripgrep to not respect gitignore and to search hidden files. But ripgrep will still skip binary files. You need `-uuu` to also ignore binary files.

I tried playing with your `rgg` function. First problem occurred when I tried it on a checkout the Linux kernel:

    $ rgg APM_RESUME
    bash: /home/andrew/rust/ripgrep/target/release/rg: Argument list too long
OK, so let's just use `xargs`:

    $ git ls-files -z | time xargs -0 rg APM_RESUME
    arch/x86/kernel/apm_32.c
    473:    { APM_RESUME_DISABLED,  "Resume timer disabled" },
    include/uapi/linux/apm_bios.h
    89:#define APM_RESUME_DISABLED  0x0d

    real    0.638
    user    0.741
    sys     1.441
    maxmem  29 MB
    faults  0
And compared to just `rg APM_RESUME`:

    $ time rg APM_RESUME
    arch/x86/kernel/apm_32.c
    473:    { APM_RESUME_DISABLED,  "Resume timer disabled" },

    include/uapi/linux/apm_bios.h
    89:#define APM_RESUME_DISABLED  0x0d

    real    0.097
    user    0.399
    sys     0.588
    maxmem  29 MB
    faults  0
So do you have an example where `git ls-files -z | xargs -0 rg ...` is faster than just `rg ...`?

Re: Ripgrep 15.0

#46

Post by burntsushi »

WJW wrote: post #42
dotancohen wrote: post #20

Earlier quoted context omitted.

Through I use rg to initiate searches, my muscle memory keeps using grep after pipes.

Huh I hadn't even realized I did that. I think grep has the "filter in pipe" spot in my head while rg has the "search recursively in all files" spot.

I did it too, even after I initially released ripgrep. At this point, I've mostly re-trained my muscle memory to use `rg` in pipelines. (Particularly because I was careful to make sure `rg` worked just like `grep` does in pipelines.)

I also find that combining `-o/--only-matching` and `-r/--replace` has replaced many of my uses of `sed` and `awk`.

Re: Ripgrep 15.0

#47

Post by oever »

burntsushi wrote: post #45
oever wrote: post #41

This week I wrote a small bash function that run ripgrep only on the files that are tracked by git: rgg() { readarray -d '' -t FILES It speeds up a lot on directories with many binary files and committed dot files. To search the dot files, -uu is needed, but that also tells ripgrep to search the binary files. On repositories with hundreds of files, the git ls-files overhead a bit large.

Can you provide a concrete example where that's faster? ripgrep should generally already be approximating `git ls-files` by respecting gitignore. Also, `-uu` tells ripgrep to not respect gitignore and to search hidden files. But ripgrep will still skip binary files. You need `-uuu` to also ignore binary files. I tried playing with your `rgg` function. First problem occurred when I tried it on a checkout the Linux ker…

A checkout of my repository [0] with many pdf and audio files (20GB) is slow with -u. These data files are normally ignored because 1) they are in .gitignore and 2) they are binary.

The repository contains CI files in .woodpecker. These are scripts that I'd normally expect to be searching in. Until a week ago I used -uu to do so, but that made rg take over 4 seconds for a search. Using -. brings the search time down to 24ms.

    git ls-files -z | time xargs -0 rg -w e23
    40ms

    rg -w. e23
    24ms

    rgg -w e23
    16ms

    rg -wuu e23
    2754ms
To reproduce this with the given repository, fill it with 20GB of binary files.

The -. flag makes this point moot though.

[0] https://codeberg.org/vandenoever/rehorse

Re: Ripgrep 15.0

#48

Post by Night_Thastus »

Ripgrep has saved me so, so much time over the years. It's become an invaluable tool, something I install the moment I start up a new system. It's essential for navigating older codebases.

My only complaint is there are a couple of characters that the -F (treat as literal) option seems to still treat as a special character needing some kind of escape - though I don't remember which ones now.

Always glad to see it keep updating!

Re: Ripgrep 15.0

#49

Post by oever »

woodruffw wrote: post #44
oever wrote: post #41

This week I wrote a small bash function that run ripgrep only on the files that are tracked by git: rgg() { readarray -d '' -t FILES It speeds up a lot on directories with many binary files and committed dot files. To search the dot files, -uu is needed, but that also tells ripgrep to search the binary files. On repositories with hundreds of files, the git ls-files overhead a bit large.

Maybe I’m missing something, but doesn’t ripgrep ignore untracked files in git by default already?

The point is to search hidden files that are tracked by git. An example is CI scripts which are stored in places like .woodpecker, .forgejo, .gitlab-ci-yml.

Re: Ripgrep 15.0

#50

Post by burntsushi »

Night_Thastus wrote: post #48

Ripgrep has saved me so, so much time over the years. It's become an invaluable tool, something I install the moment I start up a new system. It's essential for navigating older codebases. My only complaint is there are a couple of characters that the -F (treat as literal) option seems to still treat as a special character needing some kind of escape - though I don't remember which ones now. Always glad to see it kee…

> My only complaint is there are a couple of characters that the -F (treat as literal) option seems to still treat as a special character needing some kind of escape - though I don't remember which ones now.

If you have an example, I can try to explain for that specific case. But `-F/--fixed-strings` will 100% turn off any regex features in the pattern and instead will be treated as a simple literal. Where you might still need escaping is if your shell requires it.

Post reply on HN