Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unix Commands I Abuse Every Day

everythingsysadmin.com

Post reply on HN
Print view View original

11–20 of 108 posts

Re: Unix Commands I Abuse Every Day

#11

Post by jlgreco »

btilly wrote: post #6

Random note. The most commonly "abused" Unix command is cat. The name is short for "concatenate", and its intended purpose was originally to combine 2 or more files at once. Therefore every time you use it to spool one file into a pipeline, that is technically an abuse!

I have long thought that some sort of zsh completion that detects that abuse of cat and converts it into the more appropriate `< file` might be a good idea. If it did it silently it probably wouldn't be worth it but if it actually preformed the substitution in front of you then it might help users get more comfortable with the carrot syntax.

Re: Unix Commands I Abuse Every Day

#12

Post by ninetax »

andrewvc wrote: post #2

My #1 abuse is xargs -n1. A lot of people like writing bash for loops, I will try and avoid that as much as possible, xargs -n1 is the bash equivalent of a call to 'map' in a functional language. For instance, let's say you want to create thumbnails of a bunch of jpegs: find images -name "*.jpg" | xargs -n1 -IF echo F F | sed -e 's/.jpg$/_thumb.jpg/' | xargs -n2 echo convert -geometry 200x Additionally, it's fully pa…

This may be of interest to you,

http://blog.last.fm/2009/04/06/mapreduce-bash-script

Re: Unix Commands I Abuse Every Day

#13

Post by 16s »

bitwize wrote: post #3

cat foo | less Normally I'd typed something like "grep -i 'something' foo | less", and wanted to just up arrow the previous line and change the grep stuff to cat. I don't know why, it doesn't really save me anything. Maybe it's the hackerish "because I can, that's why" instinct at work.

`cat foo | less` is gratuitous. `less foo` is all you need.

Re: Unix Commands I Abuse Every Day

#14

Post by thebigpicture »

fmt 1 i don't understand his first one though. i thought both linux and bsd greps had the -H option (show filename).

here's some more:

   1. sed t file instead of cat file

   2. echo dir/*|tr '\040' '\012' instead of find dir

   3. echo dir/*|tr '\040' '\012' |sed 's/.*/program & |program2 /' > file; . file instead of xargs or find
(of course this assumes you keep filenames with spaces or other dangerous characters off your system.)

   4. same as 3. but use split to split file into parts.  then execute each part on a different cpu.

Re: Unix Commands I Abuse Every Day

#15

Post by przemoc »

andrewvc wrote: post #2

My #1 abuse is xargs -n1. A lot of people like writing bash for loops, I will try and avoid that as much as possible, xargs -n1 is the bash equivalent of a call to 'map' in a functional language. For instance, let's say you want to create thumbnails of a bunch of jpegs: find images -name "*.jpg" | xargs -n1 -IF echo F F | sed -e 's/.jpg$/_thumb.jpg/' | xargs -n2 echo convert -geometry 200x Additionally, it's fully pa…

I have a tendency to use xargs too, but often you can go with find's -exec, especially "command {} +" construct used for spitting many files at once to the given command. E.g.

    find . -iname '*.pdf' -exec pdfgrep -i keyword {} +

Re: Unix Commands I Abuse Every Day

#16

Post by sjs »

sillysaurus wrote: post #9
andrewvc wrote: post #2

My #1 abuse is xargs -n1. A lot of people like writing bash for loops, I will try and avoid that as much as possible, xargs -n1 is the bash equivalent of a call to 'map' in a functional language. For instance, let's say you want to create thumbnails of a bunch of jpegs: find images -name "*.jpg" | xargs -n1 -IF echo F F | sed -e 's/.jpg$/_thumb.jpg/' | xargs -n2 echo convert -geometry 200x Additionally, it's fully pa…

Would anyone mind doing me a favor by explaining xargs in more detail? I've tried learning it a couple times but I always seem to forget the primary situations in which it's useful. Thank you in advance!

It's most common use is to take some stuff on stdin and then use those as arguments to a command. Here is a fancy way to do `ls * `[1] using xargs:

    find . -print | xargs ls
`find` dumps the results to stdout, the pipe shuttles `find`'s stdout to `xargs` stdin, `xargs` uses its stdin as arguments for `ls`.

(Don't run this in your home directory)

[1] Pedants will realize this is actually equivalent to `ls .* *` as hidden files are found by `find`.

Re: Unix Commands I Abuse Every Day

#17

Post by Dobbs »

sillysaurus wrote: post #9
andrewvc wrote: post #2

My #1 abuse is xargs -n1. A lot of people like writing bash for loops, I will try and avoid that as much as possible, xargs -n1 is the bash equivalent of a call to 'map' in a functional language. For instance, let's say you want to create thumbnails of a bunch of jpegs: find images -name "*.jpg" | xargs -n1 -IF echo F F | sed -e 's/.jpg$/_thumb.jpg/' | xargs -n2 echo convert -geometry 200x Additionally, it's fully pa…

Would anyone mind doing me a favor by explaining xargs in more detail? I've tried learning it a couple times but I always seem to forget the primary situations in which it's useful. Thank you in advance!

Xargs takes a newline separated list and maps the list to a command.

    find ./ -name '*.log' | xargs rm
Finds all log files and map them to 'rm' commands. e.g. if it finds system.log and rails.log it will run the command `rm system.log rails.log`.

xargs will automatically do things like break up very long lists into multiple command calls so that it doesn't exceed the maximum number of arguments a command can have.

Other useful things about xargs are '-P ' which lets you run the same command in parallel. I use this with curl to do ghetto benchmarks.

The next major flag is `-n ` which changes the number of arguments per command call. e.g. `-n 1` will run the command per argument passed to xargs.

And the last flag I commonly use is `-I {}`. This sets `{}` as a variable which contains the arguments. (This also forces `-n 1`). This is useful for things like:

    find ./ -name '*.log' | xargs -I{} sh -c "if [ -f {}.gz ]; then rm {}; fi"

Re: Unix Commands I Abuse Every Day

#18

Post by morsch »

jlgreco wrote: post #11
btilly wrote: post #6

Random note. The most commonly "abused" Unix command is cat. The name is short for "concatenate", and its intended purpose was originally to combine 2 or more files at once. Therefore every time you use it to spool one file into a pipeline, that is technically an abuse!

I have long thought that some sort of zsh completion that detects that abuse of cat and converts it into the more appropriate `< file` might be a good idea. If it did it silently it probably wouldn't be worth it but if it actually preformed the substitution in front of you then it might help users get more comfortable with the carrot syntax.

The ` < file` has to appear at/near the end of the line, right? Using cat has the advantage of being able to read the line from left to right along with the data flow. I often add more piped commands to the end of a line as I refine it, while the source data remains the same. (To be fair, sometimes the opposite is true.)

Re: Unix Commands I Abuse Every Day

#19

Post by ori_b »

jlgreco wrote: post #11
btilly wrote: post #6

Random note. The most commonly "abused" Unix command is cat. The name is short for "concatenate", and its intended purpose was originally to combine 2 or more files at once. Therefore every time you use it to spool one file into a pipeline, that is technically an abuse!

I have long thought that some sort of zsh completion that detects that abuse of cat and converts it into the more appropriate `< file` might be a good idea. If it did it silently it probably wouldn't be worth it but if it actually preformed the substitution in front of you then it might help users get more comfortable with the carrot syntax.

Does it really matter that you're starting an extra process?

Re: Unix Commands I Abuse Every Day

#20

Post by lmm »

thebigpicture wrote: post #14

fmt 1 i don't understand his first one though. i thought both linux and bsd greps had the -H option (show filename). here's some more: 1. sed t file instead of cat file 2. echo dir/*|tr '\040' '\012' instead of find dir 3. echo dir/*|tr '\040' '\012' |sed 's/.*/program & |program2 /' > file; . file instead of xargs or find (of course this assumes you keep filenames with spaces or other dangerous characters off your s…

The point of the first one is he's abusing grep as cat; what he really wants is a cat that shows filenames, but since there's no such thing he uses "grep ." as a substitute.
Post reply on HN