Earlier quoted context omitted.
How long have you been using Awk?
For a few years for two clients (financial and governmental) that have a range of older controlled systems. We benefit from simple analytics that are POSIX, not python, perl, R, etc. http://www.numcommand.com/
Shell Magic: Set Operations with uniq
41–50 of 67 posts
Re: Shell Magic: Set Operations with uniq
#42 # sort a_list b_list | uniq
a
b
c
d
eRe: Shell Magic: Set Operations with uniq
#43Re: Shell Magic: Set Operations with uniq
#44Re: Shell Magic: Set Operations with uniq
#45I like shell set operations scripts, because they are quick and easy. I prefer `awk` over `uniq` and `comm` because awk tends to be faster at set ops that can skip sorting and deduplicating. Here's my script for union, intersection, etc. See README on GitHub. Suggestions welcome. https://github.com/sixarm/setop #!/bin/sh set -eu op="$1"; shift case $op in ∪|u|union|or|∨|add|addition|'+'|'|') awk '!seen[$0] {print} {s…
Re: Shell Magic: Set Operations with uniq
#46Union: Instead of
cat a_list b_list | sort | uniq
do sort -m a_list b_list | uniq
Intersection: Instead of cat a_list b_list | sort | uniq -c | grep 2\
do sort -m a_list b_list | uniq -d
Relative complement: Instead of cat a_list b_list b_list | sort | uniq -c | grep 2\
do sort -m a_list a_list b_list | uniq -u
Note the change of approach here: instead of making lines from b_list appear twice and grepping for that count, make lines from a_list appear twice and have uniq only print lines that aren't repeated.Re: Shell Magic: Set Operations with uniq
#47Earlier quoted context omitted.
More accurately, it can be used for any command which expects a file and doesn't do anything too weird in reading it (e.g. doesn't seek to the beginning and read it again) The ' $ ls
Thank you. I was just thinking of looking into how it was implemented.
http://www.tldp.org/LDP/abs/html/process-sub.html
With fish:
https://fishshell.com/docs/current/commands.html#psub
But there it only works one way:
Re: Shell Magic: Set Operations with uniq
#48I've a summary of the operations at: http://www.pixelbeat.org/cmdline.html#sets Note comm output is a bit awkward to parse, so I use another coreutils `join` command to process already sorted data
Thanks for the list, there are many useful examples. Another way to get the last date in current month (== the number of days in a month) is: : $(cal); echo $_
if whatevs; then
:
fiRe: Shell Magic: Set Operations with uniq
#49Earlier quoted context omitted.
Can also use comm To use the output of a command as input. This also works with diff and other commands.
Note that this is a bash feature; other shells may have different syntax.
Re: Shell Magic: Set Operations with uniq
#50This is a neat hack! For two input files, the intersection and relative complement can be done more straightforwardly with comm. https://en.wikipedia.org/wiki/Comm # show only items in both a and b comm -1 -2 a_list b_list # show only items unique to a comm -2 -3 a_list b_list # show only items unique to b comm -1 -3 a_list b_list
Can also use comm To use the output of a command as input. This also works with diff and other commands.
[0] https://gist.github.com/hadrianw/060944011acfcadd889d937b960...