Live data from Hacker News

Grep flags – the good stuff

zwischenzugs.com

41–50 of 53 posts

Re: Grep flags – the good stuff

#41
post #13

My favorite feature is: -P, --perl-regexp Interpret I as Perl-compatible regular expressions (PCREs). This option is experimental when combined with the -z (--null-data) option, and grep -P may warn of unimplemented features. As everything (python, Go, javascript, etc, etc) uses perl regexps now-a-days and I can never remember which things I need to escape for old gods regexp.

also combined with -o i.e.

grep -Po "Name:\K\w+"

Re: Grep flags – the good stuff

#42

It's weirdly difficult to get grep to search for fixed binary strings, with lots of gotchas if you don't understand grep internals. I still don't, but this is the best I have been able to do after knocking my forehead on three or four of said gotchas: LC_ALL=C grep -larP '\x1A\x2B\x3C\xFF'

I presume -P is a hack so that grep does the job of decoding the escape sequences? It seems your struggle here is mostly with the shell, not grep; specifically, with the fact that normal shell syntax does not recognize escape sequences other than \$, \`, \", \\, and \. Try this, which uses printf to process escape sequences.

  grep -larF "$(printf '\032\053\074\377')"
The -F flag should also make this faster as it doesn't actually need to use a regular expression engine, let alone a Perl-compatible one.

Caveats:

1) POSIX only requires printf to recognize octal escapes (\nnn, or \0nnn if using %b specifier), not hexadecimal escapes. Many implementations recognize the latter, but not Debian dash.

2) Shell command substitution strips trailing new lines from the output, so if your binary string ends in a newline you'll need to use extra tricks. E.g. S="$(printf '\032\053\074\nX')"; grep -larF "${S%X}"

3) It's probably a good idea to still specify LC_ALL=C, but because the binary string is now being passed through the shell's innards it might need to be set in the environment of the shell itself, not simply the environments of the printf and grep subcommands. (Also, technically I'm not sure if the C/POSIX locale is required to be 8-bit clean, yet, but in practice it will be.)

Bash and some other shells support an extension ($') for expanding escape sequences inline:

  grep -larF $'\x1A\x2B\x3C\xFF'
If you do any amount of shell programming--even if you only stick with Bash--it's worth spending 30 minutes reading the "Shell Command Language" chapter of the POSIX specification: https://pubs.opengroup.org/onlinepubs/9699919799/ The first few sections are the most concise resource available for explaining, step-by-step, shell parsing rules.

Re: Grep flags – the good stuff

#44
post #28

Earlier quoted context omitted.

Check out ripgrep

I'm aware of rg, ag, &c. these tools. I even wrote a clone of ag in shell using find/grep/xargs (the last one being needed to get parallelism to match ag's speed).

How did you implement gitignore filtering?

Re: Grep flags – the good stuff

#45

very frequently I want to chain grep things... I lean on the "|" operator for this, e.g. cat hello | grep foo | grep bar and it seems verbose. any tips?

Keep doing it that way, I'd say. The other replies are ways to do it without chaining grep, but they don't seem any less verbose and definitely are less obvious.

Re: Grep flags – the good stuff

#46
>The -I flag only considers text files. This radically speeds up recursive greps.

I use ripgrep when I need better speed. I've pretty much switched to ripgrep these days, but still use GNU grep when I'm answering questions on stackoverflow, reddit, etc.

>ABC flags

Good to also know about `--group-separator` and `--no-group-separator` when there are multiple non-contiguous matches. Helps to customize the separator or remove them altogether. Sadly, these options are still not explained in `man grep` on Ubuntu. You'll have to use `info grep` or the online manual to find them.

Options I use often that is not mentioned in the article:

* `-c` to count the number of matches

* `-F` for fixed string matching

* `-x` to match whole lines

* `-P` for PCRE (as mentioned in many comments here)

* `--color=auto` this is part of command name alias, so it is always used

I wrote a book as well on "GNU grep and ripgrep": https://github.com/learnbyexample/learn_gnugrep_ripgrep Free to read online.

Re: Grep flags – the good stuff

#47
post #13

My favorite feature is: -P, --perl-regexp Interpret I as Perl-compatible regular expressions (PCREs). This option is experimental when combined with the -z (--null-data) option, and grep -P may warn of unimplemented features. As everything (python, Go, javascript, etc, etc) uses perl regexps now-a-days and I can never remember which things I need to escape for old gods regexp.

Slightly pedantic, but Go uses RE2 which is subtly different than PCRE. In most common use cases, you'd probably never know.

Re: Grep flags – the good stuff

#49

I have a shell alias/function variations of which I've used for decades. This is the zsh version: function fvi { grep -rl $1 . | xargs nvim +/$1 } It greps a directory recursively and opens files which have a pattern and puts the pattern in the search buffer.

very cool!

Re: Grep flags – the good stuff

#50
post #28

Earlier quoted context omitted.

I'm aware of rg, ag, &c. these tools. I even wrote a clone of ag in shell using find/grep/xargs (the last one being needed to get parallelism to match ag's speed).

How did you implement gitignore filtering?

Very poorly.

Supporting ignore files in the root of the search is trivial.

However supporting files deeper is much harder, since find does not support this.

For the case where few directories have ignore files, it's best to find all the ignore files first and generate one find command for each.

If most directories have them then it's faster to exec find for each directory, and use maxdepth 1

In terms of me just using find by itself, I don't bother, since I find silently ignoring files to be a misfeature, and I don't use git at work anyways

Post reply on HN