21–30 of 67 posts
The cut and join commands are also useful. I'm pretty sure you could prove all basic database operations can be done using the shell.
In fact I wrote an image/tag database solution entirely in shell/coreutils. It's too disgusting to share.
This is a neat hack! For two input files, the intersection and relative complement can be done more straightforwardly with comm. https://en.wikipedia.org/wiki/Comm # show only items in both a and b comm -1 -2 a_list b_list # show only items unique to a comm -2 -3 a_list b_list # show only items unique to b comm -1 -3 a_list b_list
I think the same thing happens to a lesser degree with vim features.
I like shell set operations scripts, because they are quick and easy. I prefer `awk` over `uniq` and `comm` because awk tends to be faster at set ops that can skip sorting and deduplicating. Here's my script for union, intersection, etc. See README on GitHub. Suggestions welcome. https://github.com/sixarm/setop #!/bin/sh set -eu op="$1"; shift case $op in ∪|u|union|or|∨|add|addition|'+'|'|') awk '!seen[$0] {print} {s…
I like shell set operations scripts, because they are quick and easy. I prefer `awk` over `uniq` and `comm` because awk tends to be faster at set ops that can skip sorting and deduplicating. Here's my script for union, intersection, etc. See README on GitHub. Suggestions welcome. https://github.com/sixarm/setop #!/bin/sh set -eu op="$1"; shift case $op in ∪|u|union|or|∨|add|addition|'+'|'|') awk '!seen[$0] {print} {s…
How long have you been using Awk?
We benefit from simple analytics that are POSIX, not python, perl, R, etc. http://www.numcommand.com/
This is a neat hack! For two input files, the intersection and relative complement can be done more straightforwardly with comm. https://en.wikipedia.org/wiki/Comm # show only items in both a and b comm -1 -2 a_list b_list # show only items unique to a comm -2 -3 a_list b_list # show only items unique to b comm -1 -3 a_list b_list
Indeed comm will work much faster than uniq in this case, because it reads each file only once, and does a single pass on each file. What the blog post does (doing cat multiple times, and then counting the occurrences) is going to be much slower, although mathematically correct.
I like shell set operations scripts, because they are quick and easy. I prefer `awk` over `uniq` and `comm` because awk tends to be faster at set ops that can skip sorting and deduplicating. Here's my script for union, intersection, etc. See README on GitHub. Suggestions welcome. https://github.com/sixarm/setop #!/bin/sh set -eu op="$1"; shift case $op in ∪|u|union|or|∨|add|addition|'+'|'|') awk '!seen[$0] {print} {s…
Aaargh! "cat | sort" is an antipattern. Sort accepts a list of files as arguments. "sort [flags] file1 file2 ..." is more concise, more efficient, and is less likely to run out of space.
And for people like me who want the file name first on the line, this format works in Bash at least: outfile
This is a neat hack! For two input files, the intersection and relative complement can be done more straightforwardly with comm. https://en.wikipedia.org/wiki/Comm # show only items in both a and b comm -1 -2 a_list b_list # show only items unique to a comm -2 -3 a_list b_list # show only items unique to b comm -1 -3 a_list b_list
I love how, even after using Unix-y systems for probably 10-odd years, I still occasionally stumble upon something like this that refers to a command that I've never even heard of , and when I check (on my Mac, no less), it's right there sitting in /usr/bin. So many useful little utilities. I think the same thing happens to a lesser degree with vim features.
Other boring, but instructive reads (heavily GNU biased): diffutils [1], findutils [2], the Bash manual [3].
[0]: https://www.gnu.org/software/coreutils/manual
[1]: https://www.gnu.org/software/diffutils/manual
[2]: https://www.gnu.org/software/findutils/manual/find.html