As another user mentioned, many POSIX and/or GNU utilities havent aged well. I respect trying to stay portable, but peoples needs change over time and these tools simply havent kept up. Like the other user, I use Fd now instead: https://github.com/sharkdp/fd As well as Silver Searcher: https://github.com/ggreer/the_silver_searcher while Grep performance is pretty good, its also gotten pretty stale with regard to its…
In this case the shell hasn’t aged well. Besides the globing feature, the bash language is arcane and few can actually reliably write correct bash despite it being ubiquitous. Personally I would like to see a completely reimagined take on the shell.
A common mistake involving wildcards and the find command
81–90 of 148 posts
Re: A common mistake involving wildcards and the find command
#82https://fishshell.com/docs/current/tutorial.html#tut_wildcar...
It's rare I reach for find nowadays since switching.
Re: A common mistake involving wildcards and the find command
#83For a long time (probably since the first time I forgot to quote something and got burned, so around 40 years), I've thought that there should be some mechanism for the shell to pass in information about how each argument came about. For each argument, it would tell the program if it was supplied directly, or came from wildcard expansion. For those from wildcard expansion, it would tell the program what the wildcard…
> For those from wildcard expansion, it would tell the program what the wildcard was. Different shells have different globbing mechanisms. Why should all programs tie themselves to the mechanisms of any one particular shell? The simpler the calling mechanism for executables, the simpler it is to write them in any existing or future language. This also gives more flexibility to future shells. UNIX is pretty much desig…
I don't think my proposal would do that.
First, a clarification. When I said "For those from wildcard expansion, it would tell the program what the wildcard was" I meant the shell tells the program the full argument containing the wildcard, not just the wildcard part. E.g., if the argument was a?c which expanded to abc, the shell would tell the program it was a?c, not merely ?.
Let's say myprog takes one argument, which it expects to either be a file name or a wildcard in some pattern language that is not necessarily the same pattern language used by the shell.
Suppose myprog is invoked and it sees it has one argument, X. If the shell tells it that X did not come from shell wildcard expansion, myprog can safely use it.
If it does come from wildcard expansion and the source was Y, then there are a few cases.
1. Neither X nor Y contains any myprog wildcards. In this case, I'd have myprog go ahead and use X.
2. X does not contain any myprog wildcards, but Y does. Myprog can infer from this that it and the shell probably have overlapping wildcard languages. Report a "needs quote?" error.
3. X contains a myprog wildcard, but Y does not. Probably myprog and the shell use different wildcard languages. I'd guess its pretty unlikely that the user expects that a filename will contain a myprog wildcard, so I'd report an error in this case.
4. Both X and Y contain myprog wildcards. Go ahead an accept X, applying myprog wildcard expanstion to it. (Maybe first check that X and Y contain the same myprog wildcards, in the same order).
I'd probably add support in myprog for some optional flags to change the defaults in those cases so if you really want to do obscure things like have file names that contain myprog wildcards and use shell wildcards to provide those file names you can do it.
Re: A common mistake involving wildcards and the find command
#84https://www.tldp.org/LDP/abs/html/globbingref.html
Don't forget about globbing when making Bash scripts.
Re: A common mistake involving wildcards and the find command
#85Earlier quoted context omitted.
As another user mentioned, many POSIX and/or GNU utilities havent aged well What do you mean by that? Users have gotten more stupid and uneducated over time? I've seen "when in doubt, escape it" in several books about UNIX and shell, and escapes and wildcard expansion are one of the most beginner lessons in a lot of other materials. However, the Internet has let everyone have a voice, for better or worse, and as a re…
I have written hundreds of shell scripts, several which are hundreds of lines long. When I say they havent aged well, Im speaking from experience. When I use something like: find -name *.jpg I know what it actually does. But what it actually doesnt isnt what it should do. The current behavior returns matched files... unless no files match. Then it returns the literal string "*.jpg". That behavior is usually going to…
it going to fail if the variable has any spaces
Quoting and escaping. I mentioned that already, it's a basic part of the understanding and is not surprising at all.
Again something most people arent going to want.
You're assuming a lot. The fact that the variable substitution could expand to multiple arguments (because it's really just simple textual substitution) is in fact very powerful and useful (you can pass multiple arguments in one variable, for example), and you can always quote if you really want it to expand to one argument, the same way you would quote strings containing spaces to make them one argument. This is very consistent; observe that if $SOMETHING contains "foo bar" (without the quotes), then
grep $SOMETHING
is exactly the same as grep foo bar
but what you seem to be suggesting is that it becone grep "foo bar"
...what? Where did those quotes come from? More importantly, how do you propose to remove them? My point is, the current behaviour is simple and consistent as well as powerful and flexible, and I suspect people are just going "it doesn't work the way I thought it would" and downvoting my comment when they really haven't thought about the perfectly logical explanation for why things are the way they are. The original developers of the shell language were most certainly not stupid.Re: A common mistake involving wildcards and the find command
#86Earlier quoted context omitted.
> For those from wildcard expansion, it would tell the program what the wildcard was. Different shells have different globbing mechanisms. Why should all programs tie themselves to the mechanisms of any one particular shell? The simpler the calling mechanism for executables, the simpler it is to write them in any existing or future language. This also gives more flexibility to future shells. UNIX is pretty much desig…
> Different shells have different globbing mechanisms. Why should all programs tie themselves to the mechanisms of any one particular shell? I don't think my proposal would do that. First, a clarification. When I said "For those from wildcard expansion, it would tell the program what the wildcard was" I meant the shell tells the program the full argument containing the wildcard, not just the wildcard part. E.g., if t…
The calls of executables is done via this system call:
int execve(const char * path, char * const argv[], char * const envp[]);
How would you modify that system call signature to be able to carry that information? And what about how programs receive arguments?
int main(int argc, char * * argv)
How would you modify main()'s signature?
Take into account, the kernel has no notion that shells even exist. It has no need to know about features of particular programs in the upper layers, but now it's going to carry this information. All other programs, too. Right now, they're typically not aware that there is such a thing in the world as a shell, but now an integral part of what makes an executable an executable, its arguments, is going to come attached to this data structure that represents the use of a feature of some other program that has nothing to do with the executable itself.
What is a wildcard/glob/pattern? It's just some weird idea a program not different from any other program had. It's not particularly important. It's nothing all other executables ever needed to know about before.
Sorry I'm ranting. What I'm trying to convey is that typically software comes in layers, like an upside down pyramid, and the upper layers base themselves on the lower layers. The upper layers depend on the foundation that is the lower layer. This seems to go backwards, with the lower layer basing itself on a particular tiny insignificant piece of the upper layer. To the lower layer, that piece from the upper layer doesn't even need to exist. It can just disappear. The lower layer can run just fine without it. The kernel doesn't need the shell, and even shells don't need globbing. It's not a core feature even if it's common.
I'm fine with breaking rules and making a step towards a mess when the benefit is really worth it, but I'm not convinced this is worth it.
Listen, this article says `find -name * .jpg` is a common mistake (I put the space because of HN formatting). Yet, I don't think I've ever done that, and I don't believe I ever will. I trust the claim to an extent. When people are learning a new language (tongue), like English or Spanish, they'll make all sorts of silly grammatical mistakes, like saying "you was late" or similar. I'm sure they're very common, but as they gain experience or are a native speaker, they'll never commit those mistakes. Trust me that `find -name * .jpg` looks really wrong. It jumps out, and just like you'd never say "you was late", my fingers would never type that.
I wouldn't try to change English grammar so that "you was late" became acceptable. For whoever is making those types of mistakes, learning is a process, and that's fine.
Re: A common mistake involving wildcards and the find command
#87Earlier quoted context omitted.
For grepping, rg is faster and saner. https://github.com/BurntSushi/ripgrep On end-user systems with fast I/O, it is usually a better use of resources to have a battery-/suspend-/reboot-aware indexing system that has path/extension white/blacklists and prioritizes file monitored changes. This way, searching can happen against an optimized text search DBMS much faster than waiting for zillions of IOPS that are basical…
Do either of these tools have a "grep" mode, so that I can alias grep to it and get similar, but improved, behavior? 25 years of habit is a lot of reinforcement to overcome and I'd love to just be able to take advantage of these tools with an alias. As is I just have a grep wrapping shell script to give it some sane defaults which works fairly well.
> -u, --unrestricted ...
> Reduce the level of "smart" searching. A single -u won't respect .gitignore (etc.) files. Two -u flags will additionally search hidden files and directories. Three -u flags will additionally search binary files.
> rg -uuu is roughly equivalent to grep -r.
Re: A common mistake involving wildcards and the find command
#88As another user mentioned, many POSIX and/or GNU utilities havent aged well. I respect trying to stay portable, but peoples needs change over time and these tools simply havent kept up. Like the other user, I use Fd now instead: https://github.com/sharkdp/fd As well as Silver Searcher: https://github.com/ggreer/the_silver_searcher while Grep performance is pretty good, its also gotten pretty stale with regard to its…
Re: A common mistake involving wildcards and the find command
#89I often set the nullglob option in scripts, because it makes the handling of globs which don't match anything a bit more predictable: http://bash.cumulonim.biz/NullGlob.html There's a note at the end about how with nullglob set, ls on a glob with no matches does something surprising. This is a great illustration of how an empty list and the absence of a list are different. Sadly it's rather hard to make that distinct…
I don't know if it's the default or if it's something I've set in my config a decade or two ago but my zsh behaves like this by default (i.e. I get an error for blobs that match nothing instead of silently passing the * along). That seems much saner to me: $ find . -name *.txt zsh: no matches found: *.txt That'll teach you to quote your `find` patterns real fast...
Re: A common mistake involving wildcards and the find command
#90Earlier quoted context omitted.
> Different shells have different globbing mechanisms. Why should all programs tie themselves to the mechanisms of any one particular shell? I don't think my proposal would do that. First, a clarification. When I said "For those from wildcard expansion, it would tell the program what the wildcard was" I meant the shell tells the program the full argument containing the wildcard, not just the wildcard part. E.g., if t…
Right now, programs receive a list of strings as their arguments. What would be a concrete proposal for this X-Y mapping, taking into account that a single Y results in multiple Xes? The calls of executables is done via this system call: int execve(const char * path, char * const argv[], char * const envp[]); How would you modify that system call signature to be able to carry that information? And what about how prog…