Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A common mistake involving wildcards and the find command

blog.robertelder.org

Post reply on HN
Print view View original

101–110 of 148 posts

Re: A common mistake involving wildcards and the find command

#101

Post by rezonant »

nothrabannosir wrote: post #38

Tangential: another safeguard you can adopt is avoiding "hard delete" commands like rm and find -delete. Untrain yourself from these commands by never using them. On Mac systems, the "trash" program (brew install trash) sends files to your system trash. You can use `trash [file]` and `find .. -print0 | xargs -0 trash --`. rm is a dangerous command you should only very rarely be using. I fish something out of the tras…

> this provides the same functionality as the c program in TFA, without needing gcc.

or just using bash:

function showargs() { for x in "$@"; do echo "arg: $x"; done }

Re: A common mistake involving wildcards and the find command

#102

Post by thaumasiotes »

jrockway wrote: post #60
cellularmitosis wrote: post #53

Earlier quoted context omitted.

Why is find | grep an anti-pattern? Is eschewing memorizing the flags of one particular command in favor of using a pipeline which is much more broadly applicable really an anti-pattern? Heck, I’m at the point now where I pipe tar into gzip.

I think there is a cottage industry around telling people that they're using the interactive shell wrong. This all started with "useless use of cat", which people still bitch about on Reddit and Twitter on a regular basis, and so whenever someone finds something a little weird about UNIX, they are quick to call a person asking a question about it dumb. I think it might be stockholm syndrome, or a symptom of one's gre…

> Bad design is that find uses \n as the output record separator and that xargs uses \n as the input record separator, but \n can appear in filenames! This design can never work and will always lead to difficult-to-debug problems. So instead of fixing it

Perhaps, but it also can't be avoided. Everything can appear in filenames. We want filenames to be able to contain any character.

The most high-profile attempt to "fix" the "problem" you identify here is of course the ASCII standard, which defines several separators (0x1C - 0x1F) just for the purpose of delimiting fields with bytes that can't appear in data. Except of course nobody cares -- the solution was stillborn -- because data can contain any bytes that anyone wants it to.

> I am not very productive with PowerShell (to get a listing of files in your directory, just type "Could-We-Make-It-Even-More-Verbose -PerhapsInTheNextVersionWeWill!")

Well, you could do this by typing Get-ChildItem, but it would be both easier and more standard to type one of "gci" (what Microsoft would like you to use), "ls" (meant for those used to unix) or "dir" (for those used to dos).

Re: A common mistake involving wildcards and the find command

#103

Post by reificator »

svnpenn wrote: post #76
anonsivalley652 wrote: post #57

Earlier quoted context omitted.

For grepping, rg is faster and saner. https://github.com/BurntSushi/ripgrep On end-user systems with fast I/O, it is usually a better use of resources to have a battery-/suspend-/reboot-aware indexing system that has path/extension white/blacklists and prioritizes file monitored changes. This way, searching can happen against an optimized text search DBMS much faster than waiting for zillions of IOPS that are basical…

Yeah and its also 10 times larger https://github.com/BurntSushi/ripgrep/issues/1481

If you go on a system with grep aliased and run your single grep command, are you going to be able to tell which binary was larger?

If you look at the size of two disk images for different distros, and one has grep while the other has ripgrep, are you going to be able to tell which is which?

10x larger than grep is noise in 2020. Making that tradeoff in exchange for significantly improved performance seems reasonable.

Not to mention a large part of that difference is likely due to grep relying on libc, while ripgrep has to bundle its own runtime. And that's not even bringing up memory safety.

I'll take ripgrep at 10x the size of grep any day.

Re: A common mistake involving wildcards and the find command

#104

Post by pwdisswordfish2 »

userbinator wrote: post #85
svnpenn wrote: post #75

Earlier quoted context omitted.

I have written hundreds of shell scripts, several which are hundreds of lines long. When I say they havent aged well, Im speaking from experience. When I use something like: find -name *.jpg I know what it actually does. But what it actually doesnt isnt what it should do. The current behavior returns matched files... unless no files match. Then it returns the literal string "*.jpg". That behavior is usually going to…

You completely ignored my point, which is that none of this behaviour should be surprising or unexpected to anyone who has taken the time to think about how it all works together, and that this flexibility is the inherent way in which the UNIX command line is so powerful. it going to fail if the variable has any spaces Quoting and escaping. I mentioned that already, it's a basic part of the understanding and is not s…

So, do you prefer people have this misconception instead? https://unix.stackexchange.com/q/544189/348263

Also, for the case of passing multiple arguments to a command, just use an array:

    SOMETHING=(foo bar)
    
    grep "${SOMETHING[@]}"
It will handle arguments with spaces correctly, unlike using a string variable and splitting it on spaces. There is no reason to ever use space splitting, unless you're maintaining some legacy interface where you can only pass a string instead of an array.

Re: A common mistake involving wildcards and the find command

#108

Post by unilynx »

Dontrememberit wrote: post #90
jolmg wrote: post #86

Earlier quoted context omitted.

Right now, programs receive a list of strings as their arguments. What would be a concrete proposal for this X-Y mapping, taking into account that a single Y results in multiple Xes? The calls of executables is done via this system call: int execve(const char * path, char * const argv[], char * const envp[]); How would you modify that system call signature to be able to carry that information? And what about how prog…

Just add an ENV var containing a JSON with this metadata. There's tons of env vars that get ignored by programs that don't care about each specifically.

ENV variables are inherited so this may confuse subprocesses if they're not careful

there are workarounds to that by adding eg. the target PID, but that probably comes with its own issues

Re: A common mistake involving wildcards and the find command

#109

Post by unilynx »

thaumasiotes wrote: post #102
jrockway wrote: post #60

Earlier quoted context omitted.

I think there is a cottage industry around telling people that they're using the interactive shell wrong. This all started with "useless use of cat", which people still bitch about on Reddit and Twitter on a regular basis, and so whenever someone finds something a little weird about UNIX, they are quick to call a person asking a question about it dumb. I think it might be stockholm syndrome, or a symptom of one's gre…

> Bad design is that find uses \n as the output record separator and that xargs uses \n as the input record separator, but \n can appear in filenames! This design can never work and will always lead to difficult-to-debug problems. So instead of fixing it Perhaps, but it also can't be avoided. Everything can appear in filenames. We want filenames to be able to contain any character. The most high-profile attempt to "f…

> Perhaps, but it also can't be avoided. Everything can appear in filenames. We want filenames to be able to contain any character.

Who is "we" ? I wouldn't mind some 'sane' limits, eg UTF8, no control characters and no names starting with a dash.

Still hoping for the day that some distribution sets a mount option to enforce sane filenames by default and starts weeding out the application bugs it'll trigger..

https://dwheeler.com/essays/fixing-unix-linux-filenames.html

Re: A common mistake involving wildcards and the find command

#110

Post by madsbuch »

ulrikrasmussen wrote: post #93

I believe that programming languages should never make the meaning of a program depend on the context in which it is executed. So many obscure bugs are directly caused by such behaviors. There should be exactly one possible interpretation for a given statement, and if that cannot be executed, then the program should abort. In this case, the glob should never have been passed on to find. It should either have expanded…

One could argue that it is merely a side effect that a shell constitutes a programming language. And one could also note that find should employ the same tradition as the shell (ie. Use /*.jpg to recourse into folders)
Post reply on HN