We are stuck with egrep and fgrep (unless you like beating people) • Hacker News Board

Re: We are stuck with egrep and fgrep (unless you like beating people)

#211

Post by hnlmorg » Thu, Oct 13, 2022, 3:56 PM UTC

chaxor wrote: post #184

hnlmorg wrote: post #146

There seems to be a lot of confusion in this discussion about what the change is and how egrep and fgrep work. These are not symlinks like some have suggested but rather shell scripts. You can see the exact commit diff here: https://git.savannah.gnu.org/gitweb/?p=grep.git;a=blobdiff;f... I remember around ~10 years ago being told "you should never use `egrep` because it is slower than `grep -E`." precisely because th…

> "if concerned with performance why are you running code as a shell script" Rewrite grep in python before running - got it. :P

That's not what I suggested. I was saying if you're writing performance sensitive code with a hot loop calling `egrep` then a smarter approach might be to use a language better tuned for performance which supports regex libraries.

Shell scripts have their place and can out perform most languages if you're writing simple logic that can run across large datasets in parallel, such as

  cat very-large-file | grep "do not want" | sed -r 's/foo/bar/' > very-large-file-refactored

(please excuse the useless use of `cat`, it's there to illustrate the direction of data flow)

But in those types of scenarios the cost of $SHELL interpretation and fork() is massively outweighed by the savings of stream processing.

So my point was: if you're writing a function which $SHELL interpretation and/or fork() create enough of a performance impact where you're looking to optimize how you exec `grep -E`, then maybe it's time to investigate whether $SHELL is the right language to write your function in.

Re: We are stuck with egrep and fgrep (unless you like beating people)

#212

Post by yjftsjthsd-h » Thu, Oct 13, 2022, 3:56 PM UTC

aftbit wrote: post #138

On my system, egrep and fgrep are _already_ shell scripts, just changed to add the stupid warning: $ cat =egrep =fgrep #!/bin/sh cmd=${0##*/} echo "$cmd: warning: $cmd is obsolescent; using grep -E" >&2 exec grep -E "$@" #!/bin/sh cmd=${0##*/} echo "$cmd: warning: $cmd is obsolescent; using grep -F" >&2 exec grep -F "$@" Looking at src/egrep.sh in the git repo[1], I see that before this warning was added in 2021, the…

> cat =egrep =fgrep

What is that? I can't seem to find it by searching for " =" in `man bash`

Re: We are stuck with egrep and fgrep (unless you like beating people)

#213

Post by richard_todd » Thu, Oct 13, 2022, 3:58 PM UTC

It's interesting to think about the way these things evolved. Imagine if, over time, the `compress` utility got -gz, -bz2, -lzma, etc. flags, and `gzip`/`bzip2` were all converted to deprecated shell scripts. When is the right time to consolidate variants under one program (and deprecate the variants), and when is it better to let small utilities keep doing one thing well(TM)?

I see people talking about removing the compatibility scripts being driven by a sense of purity, but wasn't it that same sense of purity driving someone to collapse `fgrep` and `egrep` into `grep` in the first place? The sense that these are all just variants of the same goal, and thus should be flags to the same program? Why bother combining them if not to ultimately remove the "extra" programs one day?

I'm not sure what the right answer is. On one hand, I like the idea of a smaller namespace of programs with a larger set of well-documented switches. The alternate universe where `compress` covers the widely-used variants of compression and the variant utilities fell away over time sounds kind of nice. Or imagine if early UNIX had arrived at a "plugin" model, where top-level programs like grep should have pluggable regex engines which can be provided by independent projects? The culture we have, of tiny independent projects, will always make consolidation and deprecation messy events.

Re: We are stuck with egrep and fgrep (unless you like beating people)

#214

Post by hnlmorg » Thu, Oct 13, 2022, 4:01 PM UTC

vlovich123 wrote: post #186

hnlmorg wrote: post #146

There seems to be a lot of confusion in this discussion about what the change is and how egrep and fgrep work. These are not symlinks like some have suggested but rather shell scripts. You can see the exact commit diff here: https://git.savannah.gnu.org/gitweb/?p=grep.git;a=blobdiff;f... I remember around ~10 years ago being told "you should never use `egrep` because it is slower than `grep -E`." precisely because th…

> though it has already been committed to git for more than a year now and this is the first I've heard people complain about the change. This reasoning is flawed. I only noticed it a few months back on Arch. So between the time it takes the project to make a new release and it hitting Linux distros (Debian is slower and Ubuntu even moreso), don’t expect a flood of complaints to be time correlated with the change.

> This reasoning is flawed.

I wasn't making any reasoning. I was just making the observation that this change has been staged for a while now and that I'm surprised there hasn't been more noise about it before now.

> don’t expect a flood of complaints to be time correlated with the change.

I expect complaints to be correlated with whenever a blog/tweet/whatever moaning about the change happens to trend. I don't think it has much to do with when distros pick up the change because, as I also said, I expect distros to backport the old behavior. So I think the timing of any backlash is entirely dependent on the mood of the internet hive mind.

Re: We are stuck with egrep and fgrep (unless you like beating people)

#215

Post by xt00 » Thu, Oct 13, 2022, 4:02 PM UTC

I guess they backed out the “which” change before this so you wouldn’t see something silly like:

which egrep

And it would print

You should use command -v

egrep is being deprecated

/usr/..

Like jeez..

Re: We are stuck with egrep and fgrep (unless you like beating people)

#216

Post by [deleted] » Thu, Oct 13, 2022, 4:02 PM UTC

TillE wrote: post #125

cestith wrote: post #108

Earlier quoted context omitted.

Here's one example. This is in code my team inherited a long time ago, and there are many more like it. databases=`find /var/lib/mysql -type d | sed 's/\/var\/lib\/mysql\///g' | egrep -v 'mysql|test|performance|schema'`

That doesn't do anything with stderr, so it doesn't break.

[deleted]

Re: We are stuck with egrep and fgrep (unless you like beating people)

#217

Post by NegativeLatency » Thu, Oct 13, 2022, 4:04 PM UTC

thayne wrote: post #208

gatonegro wrote: post #180

Earlier quoted context omitted.

> I'd be surprised if that warning made its way to the mainstream distros. This is the first time I'm hearing about the warning, but I thought I'd run `egrep` and `fgrep` on my Arch system. Sure enough, both result in warnings: `egrep: warning: egrep/fgrep is obsolescent; using grep -E/-F` So it's made its way into Arch, at least. Though, like I said, this is the first time I'm hearing about this.

I use arch, and I had an alias for grep to expand to `egrep --color=always` in my interactive shell. I noticed real fast. Fortunately it's an easy fix to `grep -E --color=always`. sidenote: I wish that grep would allow running `grep -E -F`, and have the `-F` flag override the `-E` flag, rather than giving an error about conflicting options, so that I could have an alias like this to make extended regex the default, b…

You could write a little function that does that

Re: We are stuck with egrep and fgrep (unless you like beating people)

#218

Post by professoretc » Thu, Oct 13, 2022, 4:07 PM UTC

yjftsjthsd-h wrote: post #212

aftbit wrote: post #138

On my system, egrep and fgrep are _already_ shell scripts, just changed to add the stupid warning: $ cat =egrep =fgrep #!/bin/sh cmd=${0##*/} echo "$cmd: warning: $cmd is obsolescent; using grep -E" >&2 exec grep -E "$@" #!/bin/sh cmd=${0##*/} echo "$cmd: warning: $cmd is obsolescent; using grep -F" >&2 exec grep -F "$@" Looking at src/egrep.sh in the git repo[1], I see that before this warning was added in 2021, the…

> cat =egrep =fgrep What is that? I can't seem to find it by searching for " =" in `man bash`

In ZSH `=executable` expands to the full path of `executable`.

Re: We are stuck with egrep and fgrep (unless you like beating people)

#219

Post by manv1 » Thu, Oct 13, 2022, 4:08 PM UTC

This only affects linux. All the old AIX/SunOS/Solaris/HP-UX boxes will never get this update, so luckily it should have 0 effect on older platforms.

That makes me feel better, that all the old crap that I've forgotten about that might be running in production somewhere will be fine.

That said, this is a ridiculous move. The only practical effect of this change is to potentially break millions of scripts, all for an ideal that, to be frank, nobody gives a shit about.

Re: We are stuck with egrep and fgrep (unless you like beating people)

#220

Post by kccqzy » Thu, Oct 13, 2022, 4:08 PM UTC

aftbit wrote: post #138

On my system, egrep and fgrep are _already_ shell scripts, just changed to add the stupid warning: $ cat =egrep =fgrep #!/bin/sh cmd=${0##*/} echo "$cmd: warning: $cmd is obsolescent; using grep -E" >&2 exec grep -E "$@" #!/bin/sh cmd=${0##*/} echo "$cmd: warning: $cmd is obsolescent; using grep -F" >&2 exec grep -F "$@" Looking at src/egrep.sh in the git repo[1], I see that before this warning was added in 2021, the…

If you have interacted with Paul Eggert in real life (I have; he's a professor at UCLA), you would've known that he is a man that values purity over practicality. Not surprising at all.