Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A common mistake involving wildcards and the find command

blog.robertelder.org

Post reply on HN
Print view View original

131–140 of 148 posts

Re: A common mistake involving wildcards and the find command

#131

Post by burntsushi »

svnpenn wrote: post #76
anonsivalley652 wrote: post #57

Earlier quoted context omitted.

For grepping, rg is faster and saner. https://github.com/BurntSushi/ripgrep On end-user systems with fast I/O, it is usually a better use of resources to have a battery-/suspend-/reboot-aware indexing system that has path/extension white/blacklists and prioritizes file monitored changes. This way, searching can happen against an optimized text search DBMS much faster than waiting for zillions of IOPS that are basical…

Yeah and its also 10 times larger https://github.com/BurntSushi/ripgrep/issues/1481

Yes. If that disk space is more important to you than correctness, performance and better Unicode support, then yeah, you should definitely keep using the silver searcher.

I gave more details in a response on that issue: https://github.com/BurntSushi/ripgrep/issues/1481#issuecomme...

Re: A common mistake involving wildcards and the find command

#132

Post by burntsushi »

eikenberry wrote: post #62
anonsivalley652 wrote: post #57

Earlier quoted context omitted.

For grepping, rg is faster and saner. https://github.com/BurntSushi/ripgrep On end-user systems with fast I/O, it is usually a better use of resources to have a battery-/suspend-/reboot-aware indexing system that has path/extension white/blacklists and prioritizes file monitored changes. This way, searching can happen against an optimized text search DBMS much faster than waiting for zillions of IOPS that are basical…

Do either of these tools have a "grep" mode, so that I can alias grep to it and get similar, but improved, behavior? 25 years of habit is a lot of reinforcement to overcome and I'd love to just be able to take advantage of these tools with an alias. As is I just have a grep wrapping shell script to give it some sane defaults which works fairly well.

Author of ripgrep here.

> As is I just have a grep wrapping shell script to give it some sane defaults which works fairly well.

Yes, that's what I did for about ten years before I wrote ripgrep. :-)

> Do either of these tools have a "grep" mode, so that I can alias grep to it and get similar, but improved, behavior?

Another commenter already mentioned `rg -uuu`, and that's pretty much the right answer. In a large number of cases, if you `alias grep=rg`, then most things will continue to work as you expect. If you really do not want any kind of smart filtering, then yes, you'll want `alias grep="rg -uuu"`. But otherwise, things like `command ... | rg pattern` and `rg pattern file ...` will continue to work just like grep.

ripgrep also tries to use the same names for flags as grep, wherever possible. So you'll find a lot of overlap there too.

Please note that ripgrep is not and is not intended to be a drop-in replacement. But I didn't go out of my way to be incompatible with grep either. So a lot of it should be quite familiar!

Re: A common mistake involving wildcards and the find command

#133

Post by useerup »

jrockway wrote: post #60
cellularmitosis wrote: post #53

Earlier quoted context omitted.

Why is find | grep an anti-pattern? Is eschewing memorizing the flags of one particular command in favor of using a pipeline which is much more broadly applicable really an anti-pattern? Heck, I’m at the point now where I pipe tar into gzip.

I think there is a cottage industry around telling people that they're using the interactive shell wrong. This all started with "useless use of cat", which people still bitch about on Reddit and Twitter on a regular basis, and so whenever someone finds something a little weird about UNIX, they are quick to call a person asking a question about it dumb. I think it might be stockholm syndrome, or a symptom of one's gre…

> just type "Could-We-Make-It-Even-More-Verbose -PerhapsInTheNextVersionWeWill!"

PowerShell can be as terse as bash. Just use the aliases (which are actually real aliases in PowerShell), leverage default parameters and shorten parameter names (or use their aliases) to just use enough character to disambiguate.

To get a file listing of your directory, simply type

    ls
If you want to know why this works, type

    help ls
and PowerShell will answer you with the help on "Get-ChildItem".

You could also type this to explain the `ls` command:

    gcm ls
And PowerShell will answer

    CommandType     Name                                               Version    Source
    -----------     ----                                               -------    ------
    Alias           ls -> Get-ChildItem
`gcm` is itself an alias for Get-Command - a command that gets information about a command. Executing `gcm gcm` will produce this output:

    CommandType     Name                                               Version    Source
    -----------     ----                                               -------    ------
    Alias           gcm -> Get-Command
Incidentally, why does `find` have a `--delete` option in the first place? That's not very do one thing only and do it well. `find` should search/find and do that well. Why is it also a file-deleter?

Going back to the example from TFA, in PowerShell you would delete files not matching a pattern using this command:

    ls -ex *.py -rec -file | rm
Or, if you do want it verbose, using full names instead of aliases and shortened parameter names:

    Get-ChildItem -Exclude *.py -Recurse -File | Remove-Item

Re: A common mistake involving wildcards and the find command

#134

Post by Aardwolf »

svnpenn wrote: post #16

As another user mentioned, many POSIX and/or GNU utilities havent aged well. I respect trying to stay portable, but peoples needs change over time and these tools simply havent kept up. Like the other user, I use Fd now instead: https://github.com/sharkdp/fd As well as Silver Searcher: https://github.com/ggreer/the_silver_searcher while Grep performance is pretty good, its also gotten pretty stale with regard to its…

Imho also anything that outputs or uses multiples of 512 bytes (blocks) by default

Sure, disks may have sectors and there may be some use cases, but we use bytes, kilobytes, kibibytes, etc now for data sizes.

Re: A common mistake involving wildcards and the find command

#135

Post by Hamuko »

JosefAssad wrote: post #123
Hamuko wrote: post #115

Earlier quoted context omitted.

I do for .pyc files. find . -name "*.pyc" -delete

Is there a reason you prefer that rather than having something like *.py[cod] in your .gitignore?

Yeah, the reason is that the Python interpreter doesn't give a shit about .gitignore.

Re: A common mistake involving wildcards and the find command

#136

Post by scolby33 »

porker wrote: post #120
userbinator wrote: post #74

Earlier quoted context omitted.

As another user mentioned, many POSIX and/or GNU utilities havent aged well What do you mean by that? Users have gotten more stupid and uneducated over time? I've seen "when in doubt, escape it" in several books about UNIX and shell, and escapes and wildcard expansion are one of the most beginner lessons in a lot of other materials. However, the Internet has let everyone have a voice, for better or worse, and as a re…

I don't understand escaping properly. Why, on some machines (all varying distros of Ubuntu) does find -name *.xml return nothing and I have to use find -name '*.xml' whilst on other machines the first command returns results? The shell is always bash, it's a stock install - so why the variation? I get that it should always be quoted, but not why it sometimes works when unquoted and other times doesn't.

Perhaps `shopt -s nullglob` has been set?

“If set, Bash allows filename patterns which match no files to expand to a null string, rather than themselves.” [1]

[1] https://www.gnu.org/software/bash/manual/html_node/The-Shopt...

Re: A common mistake involving wildcards and the find command

#137

Post by bmn__ »

porker wrote: post #120
userbinator wrote: post #74

Earlier quoted context omitted.

As another user mentioned, many POSIX and/or GNU utilities havent aged well What do you mean by that? Users have gotten more stupid and uneducated over time? I've seen "when in doubt, escape it" in several books about UNIX and shell, and escapes and wildcard expansion are one of the most beginner lessons in a lot of other materials. However, the Internet has let everyone have a voice, for better or worse, and as a re…

I don't understand escaping properly. Why, on some machines (all varying distros of Ubuntu) does find -name *.xml return nothing and I have to use find -name '*.xml' whilst on other machines the first command returns results? The shell is always bash, it's a stock install - so why the variation? I get that it should always be quoted, but not why it sometimes works when unquoted and other times doesn't.

> so why the variation?

Difference between dash and bash?

Shell options? `man shopt` and grep for glob

Re: A common mistake involving wildcards and the find command

#138

Post by bmn__ »

fit2rule wrote: post #124
svnpenn wrote: post #16

As another user mentioned, many POSIX and/or GNU utilities havent aged well. I respect trying to stay portable, but peoples needs change over time and these tools simply havent kept up. Like the other user, I use Fd now instead: https://github.com/sharkdp/fd As well as Silver Searcher: https://github.com/ggreer/the_silver_searcher while Grep performance is pretty good, its also gotten pretty stale with regard to its…

For me, Silver Searcher often fails to find matches in files that 'grep' will, easily. I've not been able to determine why this is, but it happens often enough that I consider SS to be highly unreliable, even if its fast. So, it seems that there are still some issues to be sorted here.

This claim needs adequate proof, please provide an example. Otherwise the HN reader has to assume you made a user error.

Re: A common mistake involving wildcards and the find command

#139

Post by School-Cotton »

erikbye wrote: post #116
School-Cotton wrote: post #30

My instant reaction to the example was “that won’t work; you’re shell will say something like ‘no matches’”. Using an unescaped star in a find command never works for me, which is a lot better than it sometimes working and sometimes breaking! Reading the article and the comments, it seems like bash doesn’t do this? I suppose it’s one nice thing about oh-my-zsh, whose default confit I use almost unchanged.

It's a default in zsh, nevermind oh-my-zsh.

Thanks; I didn’t know whether it was or not while I was writing my comment, and it was difficult to check on mobile, so I went with what I knew for sure.

Glad to know it’s a stock zsh default, too!

Re: A common mistake involving wildcards and the find command

#140

Post by UncleMeat »

userbinator wrote: post #85
svnpenn wrote: post #75

Earlier quoted context omitted.

I have written hundreds of shell scripts, several which are hundreds of lines long. When I say they havent aged well, Im speaking from experience. When I use something like: find -name *.jpg I know what it actually does. But what it actually doesnt isnt what it should do. The current behavior returns matched files... unless no files match. Then it returns the literal string "*.jpg". That behavior is usually going to…

You completely ignored my point, which is that none of this behaviour should be surprising or unexpected to anyone who has taken the time to think about how it all works together, and that this flexibility is the inherent way in which the UNIX command line is so powerful. it going to fail if the variable has any spaces Quoting and escaping. I mentioned that already, it's a basic part of the understanding and is not s…

Our tools should be dead simple whenever possible rather than "if you make mistakes then git gud".
Post reply on HN