91–100 of 135 posts
I scanned until I saw `ls | egrep '.*_test\.(py|cc)' | xargs -d $'\n' -- rm`, and then stopped. This is a terrible idea[1][2]. [1] https://mywiki.wooledge.org/ParsingLs [2] https://unix.stackexchange.com/q/128985/3645
Since the blog author is commenting here, you have this statement part way down your blog: > That is, grep doesn't support an analogous -0 flag. However, the GNU grep variant does have an analogous flag: -z, --null-data Treat the input as a set of lines, each terminated by a zero byte (the ASCII NUL character) instead of a newline. Like the -Z or --null option, this option can be used with commands like sort -z to pr…
Ah cool, I didn't know that! I'll update the blog post. (What a cacophony of flags) Edit: It seems that grep -0 isn't taken for something else and they should have used it for consistency? The man page says it's meant to be used with find -print0, xargs -0, perl -0, and sort -z (another inconsistency)
Earlier quoted context omitted.
Yeah, although I use the parentheses mostly because I like how it reads. And that process substitution trick is important too. I think the redirection can come first, though (not at a computer to test):
This is not POSIX compliant though.
Earlier quoted context omitted.
Ah cool, I didn't know that! I'll update the blog post. (What a cacophony of flags) Edit: It seems that grep -0 isn't taken for something else and they should have used it for consistency? The man page says it's meant to be used with find -print0, xargs -0, perl -0, and sort -z (another inconsistency)
It's best to give up on any kind of consistency between command options. Any project is free to do anything it wants, and they all do. Someone is eventually going to come up with standard N+1[1] which does things consistently, but they are going to have to either recreate a bazillion tools or create some sort of huge translation framework configuration on top of existing tools to get there. And even then it'll take l…
> A lobste.rs user asked why you would use find | xargs rather than find -exec. The answer is that it can be much faster. If you’re trying to rm 10,000 files, you can start one process instead of 10,000 processes! Fair enough, but I still favor find -exec . I find it generally less error prone, and it's never been so slow that I wished I had instead used xargs. Also, if you're specifically using -exec rm with find, y…
You can also use `find -exec` with `'+'` instead of `';'` as the terminator. This will call `rm` on all of the found files in one call.
after spending a bit of time reading the man page for find, i rarely use xargs any more. find is pretty good.
tangent:
another instance i've seen where spawning many processes can lead to bad performance is in bash scripts for git pre-recieve hooks, to scan and validate the commit message of a range of commits before accepting them. it is pretty easy to cobble together some loop in a bash script that executes multiple processes _per commit_. that's fine for typical small pushes of 1-20 commits -- but if someone needs to do serious graph surgery and push a branch of 1000 - 10,000 commits that can can cause very long running times -- and more seriously, timeouts, where the entire push gets rejected as the pre-receive script takes too long. a small program using the libgit2 API can do the same work at the cost of a single process, although then you have the fun of figuring out how to build, install and maintain binary git pre-receive hooks.
Earlier quoted context omitted.
The nagware prompts of parallel are so objectionable that I will do a lot of things to avoid using it at all. So pretentious!
It's also written in Perl!
Of xargs, for, and while, I have limited myself to while. It's more typing everytime but saves me from having to remember so many quirks of each command. cat input.file | ... | while read -r unit; do ${unit}; done | ... between 'while read -r unit' and 'while IFS= read -r unit' I can probably handle 90% of the cases. (maybe I should always use IFS since I tend to forget the proper way to use it).
I suspect I'll really like your way of doing things, but an example would be very handy.
Earlier quoted context omitted.
This is not POSIX compliant though.
These days bash and/or zsh are available nearly every place I care about, so I find POSIX compliance to be much less relevant.
Since the blog author is commenting here, you have this statement part way down your blog: > That is, grep doesn't support an analogous -0 flag. However, the GNU grep variant does have an analogous flag: -z, --null-data Treat the input as a set of lines, each terminated by a zero byte (the ASCII NUL character) instead of a newline. Like the -Z or --null option, this option can be used with commands like sort -z to pr…
Ah cool, I didn't know that! I'll update the blog post. (What a cacophony of flags) Edit: It seems that grep -0 isn't taken for something else and they should have used it for consistency? The man page says it's meant to be used with find -print0, xargs -0, perl -0, and sort -z (another inconsistency)