Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We are stuck with egrep and fgrep (unless you like beating people)

utcc.utoronto.ca

Post reply on HN
Print view View original

301–310 of 354 posts

Re: We are stuck with egrep and fgrep (unless you like beating people)

#301

Post by chasil »

chasil wrote: post #263
hinkley wrote: post #235

Earlier quoted context omitted.

Different behavior based on argv[0] was first brought to my attention when I discovered that /bin/sh was a symlink on some Linux systems. Bash has a Bourne shell compatibility mode.

Bash actually has a POSIX compatibility mode. There is a great deal in POSIX that was not in Bourne, native arithmetic expressions being the first to come to mind, then new-style command substitution. https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V... All of the POSIX utility standards, including the grep variants, can be found in the URL's parent directory: https://pubs.opengroup.org/onlinepubs/96999197…

I just checked OpenBSD, and I find that there are 6 total links in /usr/bin: egrep, fgrep, grep, zegrep, zfgrep, and zgrep.

The OpenBSD package is superior.

The GNU gzip package includes a zgrep shell script that is adapted from work by "Charles Levert " - this is similarly adapted for bzgrep and xzgrep.

The OpenBSD implementation will have superior performance for zgrep, because it is native C.

  rebel$ ls -li /usr/bin/*grep 
  466612 -r-xr-xr-x  6 root  bin  31520 Apr 11  2022 /usr/bin/egrep
  466612 -r-xr-xr-x  6 root  bin  31520 Apr 11  2022 /usr/bin/fgrep
  466612 -r-xr-xr-x  6 root  bin  31520 Apr 11  2022 /usr/bin/grep
  466711 -r-xr-xr-x  2 root  bin  15288 Apr 11  2022 /usr/bin/pgrep
  466612 -r-xr-xr-x  6 root  bin  31520 Apr 11  2022 /usr/bin/zegrep
  466612 -r-xr-xr-x  6 root  bin  31520 Apr 11  2022 /usr/bin/zfgrep
  466612 -r-xr-xr-x  6 root  bin  31520 Apr 11  2022 /usr/bin/zgrep

Re: We are stuck with egrep and fgrep (unless you like beating people)

#302

Post by romeoblade »

teddyh wrote: post #239
radford-neal wrote: post #229

Beyond the inadvisability of making a breaking change for no reason, it's worth noting that deprecating fgrep is actually positively undesirable. Use of fgrep should be encouraged . The reason is that many times people want to match a literal string. This is best done with, for example, "fgrep [a] Of course, one could get in the habit of using grep -F when intending the pattern to be just a literal string. Or one cou…

If we are talking about interactive use, you could always use an alias. But if we are talking about scripts , you should at least use "fgrep -- [a] If you don’t do this, the script will fail spectacularly the day when the string happens to start with a hyphen (-).

I thought I was the only one who perfected long options! Are you me?

I've had coworkers call me out (not rude, just "hey you know you just use -l... instead of --longopt") on calls because I always use long options when available. I use the hyphen explanation all the time as I've ran into it a few times.

I also prefer CLI applications that are designed to use the "=" for arguments with long options. Applications which don't use "=" or respect it, irk me because the it's ambiguous... "Is that argument an argument or sub command" when looking through history.

Re: We are stuck with egrep and fgrep (unless you like beating people)

#303

Post by sprremix »

yjftsjthsd-h wrote: post #212
aftbit wrote: post #138

On my system, egrep and fgrep are _already_ shell scripts, just changed to add the stupid warning: $ cat =egrep =fgrep #!/bin/sh cmd=${0##*/} echo "$cmd: warning: $cmd is obsolescent; using grep -E" >&2 exec grep -E "$@" #!/bin/sh cmd=${0##*/} echo "$cmd: warning: $cmd is obsolescent; using grep -F" >&2 exec grep -F "$@" Looking at src/egrep.sh in the git repo[1], I see that before this warning was added in 2021, the…

> cat =egrep =fgrep What is that? I can't seem to find it by searching for " =" in `man bash`

It's a zsh thing.

14.7.3 ‘=’ expansion

If a word begins with an unquoted ‘=’ and the EQUALS option is set, the remainder of the word is taken as the name of a command. If a command exists by that name, the word is replaced by the full pathname of the command.

From: https://zsh.sourceforge.io/Doc/Release/Expansion.html#g_t_00...

Re: We are stuck with egrep and fgrep (unless you like beating people)

#304

Post by mike_hock »

tannhaeuser wrote: post #191
ratmice wrote: post #177

Earlier quoted context omitted.

The gnu coding standards has this to say: https://www.gnu.org/prep/standards/standards.html#index-beha... Please don’t make the behavior of a utility depend on the name used to invoke it. It is useful sometimes to make a link to a utility with a different name, and that should not change what it does. The next section provides some reasoning: Providing valid information in argv[0] is a convention, not guaranteed. Wel…

Thanks for the link, but I find that reasoning entirely unconvincing: The ability to interpret argv[0] as a way to avoid exec is traded for hypothetical linking - but wherever linking is used (such as for idk a mail filter or other configurable executable) that's the place where a wrapper shell script could be used instead. And avoiding improper launching of egrep/fgrep from a program that doesn't bother to build arg…

I find the reasoning convincing. It's the principle of least surprise. And a bunch of other principles. I don't want a program to care what it's executable name is, if it's being invoked with a relative path, absolute path, through a symlink or hardlink or anything else.

Of course, there are exceptions to the rule, like busybox. The point is to have uniform, predictable behavior across the ecosystem and not have every other program have its own weird idiosyncrasies.

That being said, the egrep/fgrep legacy is precisely such a case where it makes sense to make an exception. It's a decades-old legacy and grep isn't just any program, it's part of Using the Shell 101.

Re: We are stuck with egrep and fgrep (unless you like beating people)

#305

Post by int_19h »

Pxtl wrote: post #94

I do wish that there was a standard pipeline for warnings, so stuff like this didn't have to go into error (bad) or output (worse). Powershell has such a feature but of course since it's not at the OS/Posix level then support is spotty.

Why is it bad for it to go stderr? It's supposed to be used for all kinds of app meta-output, not just errors. If you want to actually check if the command failed, that's what exit codes are for.

I did run into some Node.js code that assumed that anything printed out to stderr is a fatal error - but that's just people making wrong assumptions, not using the interface as intended.

Re: We are stuck with egrep and fgrep (unless you like beating people)

#306

Post by int_19h »

cestith wrote: post #297
TillE wrote: post #125

Earlier quoted context omitted.

That doesn't do anything with stderr, so it doesn't break.

It does output to STDERR an extra warning.

But that doesn't actually break your script, because backticks only capture stdout.

Re: We are stuck with egrep and fgrep (unless you like beating people)

#307

Post by int_19h »

cesarb wrote: post #28
tuetuopay wrote: post #14

Earlier quoted context omitted.

It's more than likely the warning will be printed to stderr, not out, so there will be no impact on the actual work done.

It's not unusual in shell scripts to combine stderr with stdout by using "2>&1" or similar.

It is, however, very unusual to do so and then try to parse the output. Aside from compilers, what other CLI tools make any guarantees wrt what they print to stderr?

Re: We are stuck with egrep and fgrep (unless you like beating people)

#308

Post by cryptonector »

hnlmorg wrote: post #146

There seems to be a lot of confusion in this discussion about what the change is and how egrep and fgrep work. These are not symlinks like some have suggested but rather shell scripts. You can see the exact commit diff here: https://git.savannah.gnu.org/gitweb/?p=grep.git;a=blobdiff;f... I remember around ~10 years ago being told "you should never use `egrep` because it is slower than `grep -E`." precisely because th…

The maintainers' fears about using argv[0] to select behavior are unfounded.

Anyone exec()'ing grep with an alternate argv[0] can just use (or not use) the -E/-F options to get the behavior they want.

It's very simple. There is just no good reason for egrep or fgrep to be shell scripts or wrapper programs of any kind. Even if the were a good reason for having a wrapper program, the wrapper program could be a very tiny C (or even assembly) program that just exec()s grep with -E or -F added, thus avoiding any additional fork() calls (though there would still be one additional exec() call).

But, really, argv[0] dispatching is plenty good enough.

This warning, however, is much too annoying.

Plus, argv[0] dispatching has worked for every other Unix and Unix-like OS of note for 4+ decades! What's so special about GNU in this regard? The answer is clearly: nothing.

Re: We are stuck with egrep and fgrep (unless you like beating people)

#309

Post by naikrovek »

cryptonector wrote: post #248
raverbashing wrote: post #241

Earlier quoted context omitted.

> But it is always possible to launch the program and give a nonsensical value in argv[0]. Well, if you're being a smart@ss and doing that, don't expect it not to break later. Putting a weird value there and hoping it won't break is as stupid as calling the program with wrong options and hoping it will work.

Exactly.

the name of an executable is like a global variable the instant it is used as data; it can be changed at any time by someone else and then behavior changes.

names are not data. don't make them data.

names are names.

Re: We are stuck with egrep and fgrep (unless you like beating people)

#310

Post by hnlmorg »

jraph wrote: post #300
hnlmorg wrote: post #211

Earlier quoted context omitted.

That's not what I suggested. I was saying if you're writing performance sensitive code with a hot loop calling `egrep` then a smarter approach might be to use a language better tuned for performance which supports regex libraries. Shell scripts have their place and can out perform most languages if you're writing simple logic that can run across large datasets in parallel, such as cat very-large-file | grep "do not w…

> please excuse the useless use of `cat`, it's there to illustrate the direction of data flow Note that you can write Instead of cat very-large-file | To avoid a useless use of cat and keep the direction. Not that I care very much though. Using cat is less surprising to most people.

> Using cat is less surprising to most people.

Exactly :)

STDIN redirection is a neat trick though so definitely worth highlighting. But it wouldn’t have helped with readability in my particular example.

Post reply on HN