Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unix Commands I Abuse Every Day

everythingsysadmin.com

Post reply on HN
Print view View original

81–90 of 108 posts

Re: Unix Commands I Abuse Every Day

#81

Post by ralph »

njs12345 wrote: post #46
veyron wrote: post #27

Earlier quoted context omitted.

it doesnt matter for a file of size 1kb. For a file of size 10Gb, every process matters. For the downvoters: please time how long it takes to do something like `cat $file | awk '{print $1}' ` and `awk <$file '{print $1}'`

So the two are different because awk's call to read() is effectively the same as a read directly from a file, whereas copying is taking place through the pipe with the pipeline approach?

The difference between

    cat file | foo
    foo
assuming foo only reads stdin so `foo file' isn't possible, is that with the latter the shell will open file for reading on file descriptor 0 (stdin) before execing foo and the only cost is the read(2)s that foo does directly from file.

With the needless cat we have cat having to read the bytes and then write(2) them whereupon foo reads them as before. So the number of system calls goes from R to R+W+R assuming all reads and writes use the same block size and more byte copying may be required.

Re: Unix Commands I Abuse Every Day

#82

Post by ralph »

shorter wrote: post #67
btilly wrote: post #6

Random note. The most commonly "abused" Unix command is cat. The name is short for "concatenate", and its intended purpose was originally to combine 2 or more files at once. Therefore every time you use it to spool one file into a pipeline, that is technically an abuse!

I've sometimes heard it's "catenate" not "concatenate". Do they mean the same thing? As for cat, the utility, I'm afraid we'll never stop seeing people doing cat file|prog1|prog2 even when it makes absolutely no sense whatsoever. If it did, I might as well do cat file|cat|prog|cat|prog2|cat - I mean, why not? It "looks nicer" than prog file|prog2 or prog Programmers love their cats.

It's most definitely catenate. I understand catenate to mean chain and concatenate to be to chain together. Since "cat foo bar xyzzy" doesn't modify the files to join them in any way I don't think they're chained together.

Besides, ken & Co. aren't daft. con would be short for concatenate. :-)

Re: Unix Commands I Abuse Every Day

#83

Post by BrandonM »

_ZeD_ wrote: post #60
BrandonM wrote: post #53

Earlier quoted context omitted.

Why do that when a loop is clearer, safer, and shorter? find images -name '*.jpg' | while read jpg; do convert -geometry 200x "$jpg" "$(echo "$jpg" | sed 's/.jpg$/_thumb.jpg/')"; done This version works even when there are spaces in a filename, whereas yours will break.

false $ ls -Ql totale 4 -rw-r--r-- 1 zed users 33 set 6 07:30 " spaces " $ $ find -name '*spaces*' | while read text; do cat "$text"; done cat: ./ spaces: File o directory non esistente $ $ find -name '*spaces*' -print0 | xargs -0 cat while read is broken with spaces $

That is a good point, but it is a much more degenerate case than the more common internal space.

Re: Unix Commands I Abuse Every Day

#84

Post by BrandonM »

sjs wrote: post #16
sillysaurus wrote: post #9

Earlier quoted context omitted.

Would anyone mind doing me a favor by explaining xargs in more detail? I've tried learning it a couple times but I always seem to forget the primary situations in which it's useful. Thank you in advance!

It's most common use is to take some stuff on stdin and then use those as arguments to a command. Here is a fancy way to do `ls * `[1] using xargs: find . -print | xargs ls `find` dumps the results to stdout, the pipe shuttles `find`'s stdout to `xargs` stdin, `xargs` uses its stdin as arguments for `ls`. (Don't run this in your home directory) [1] Pedants will realize this is actually equivalent to `ls .* *` as hidd…

Maybe you'll call me a pedant, but you should be aware that `find .` is not equivalent to `ls .* *`. The find command starts at the indicated directories (. in this case) and lists each file and directory within it, recursing into subdirectories. You can use things like -type, -[i]name, and -mtime to filter the results, as well as -mindepth and -maxdepth to constrain the traversal.

Note also that "-print" is the default command for find, so you can leave it off. Other commands include -print0 (NUL-delimited instead of newline-delimited) and -exec.

Re: Unix Commands I Abuse Every Day

#85

Post by krzyk »

przemoc wrote: post #15
andrewvc wrote: post #2

My #1 abuse is xargs -n1. A lot of people like writing bash for loops, I will try and avoid that as much as possible, xargs -n1 is the bash equivalent of a call to 'map' in a functional language. For instance, let's say you want to create thumbnails of a bunch of jpegs: find images -name "*.jpg" | xargs -n1 -IF echo F F | sed -e 's/.jpg$/_thumb.jpg/' | xargs -n2 echo convert -geometry 200x Additionally, it's fully pa…

I have a tendency to use xargs too, but often you can go with find's -exec, especially "command {} +" construct used for spitting many files at once to the given command. E.g. find . -iname '*.pdf' -exec pdfgrep -i keyword {} +

Upvoted just for mentioning that there is such thing as pdfgrep, thanks :)

Re: Unix Commands I Abuse Every Day

#86

Post by Nick_C »

AgentConundrum wrote: post #40
raldi wrote: post #31

You can do "head -n 0" on Linux to mean "all lines". No you can't.

`head -n 99999` seems like a weird way to do it anyway. Wouldn't it make more sense to do `tail -n +1`? The output is the same from both commands, but `tail` doesn't require you to assume arbitrary limits. Honest question, btw. I'm relatively inexperienced with Linux, and I certainly haven't used BSD. I'd appreciate any critiques you may have to offer.

> Wouldn't it make more sense to do `tail -n +1`

Yes (or perhaps tail -n +0 as that is idiomatic, which makes it clear to anyone what you are intending).

Re: Unix Commands I Abuse Every Day

#87

Post by lloeki »

ralph wrote: post #75
btilly wrote: post #6

Random note. The most commonly "abused" Unix command is cat. The name is short for "concatenate", and its intended purpose was originally to combine 2 or more files at once. Therefore every time you use it to spool one file into a pipeline, that is technically an abuse!

No, cat(1) stands for catenate, not concatenate. http://en.wiktionary.org/wiki/catenate

hmmm, cat(1) mentions interesting flags, and tac, which comes in handy at times.

Re: Unix Commands I Abuse Every Day

#88

Post by ishkur101 »

ralph wrote: post #82
shorter wrote: post #67

Earlier quoted context omitted.

I've sometimes heard it's "catenate" not "concatenate". Do they mean the same thing? As for cat, the utility, I'm afraid we'll never stop seeing people doing cat file|prog1|prog2 even when it makes absolutely no sense whatsoever. If it did, I might as well do cat file|cat|prog|cat|prog2|cat - I mean, why not? It "looks nicer" than prog file|prog2 or prog Programmers love their cats.

It's most definitely catenate. I understand catenate to mean chain and concatenate to be to chain together. Since "cat foo bar xyzzy" doesn't modify the files to join them in any way I don't think they're chained together. Besides, ken & Co. aren't daft. con would be short for concatenate. :-)

From the man page cat doc1 doc2 > doc.all concatenates the files doc1 and doc2 and writes the result to doc.all

Re: Unix Commands I Abuse Every Day

#89

Post by dave1010uk »

stefanu wrote: post #48

Stop watch: time read press enter to read elapsed time. If you write your activity in the prompt and repeat it for multiple activities, you have a nice time log. You can then just copy&paste it from terminal.

I have the time (H:m:s) in my prompt. That way I can easily time commands and I can also find things more easily in my scroll buffer.

Re: Unix Commands I Abuse Every Day

#90

Post by ralph »

ishkur101 wrote: post #88
ralph wrote: post #82

Earlier quoted context omitted.

It's most definitely catenate. I understand catenate to mean chain and concatenate to be to chain together. Since "cat foo bar xyzzy" doesn't modify the files to join them in any way I don't think they're chained together. Besides, ken & Co. aren't daft. con would be short for concatenate. :-)

From the man page cat doc1 doc2 > doc.all concatenates the files doc1 and doc2 and writes the result to doc.all

I think I'm missing your point. doc[12] aren't changed.
Post reply on HN