Why Learn Awk? (2016)

Post by jimbokun » Tue, Jan 21, 2020, 10:36 PM UTC

dbro wrote: post #102

berkes wrote: post #72

Last week I threw out AWK and replaced it with Ruby (Could've been Python, Perl or PHP even). Because AWK is not suited for CSV. Please prove me wrong! I had to parse 9million lines. Some of which contain "quoted records", others, same column, are unquoted. Some contain comma's, in the fields, most don't. CSV is like that: more like a guideline than actual sense. Two hours of googling and hacking later, I gave up and…

To simplify working with CSV data using command line tools, I wrote csvquote ( https://github.com/dbro/csvquote ). There are some examples on that page that show how it works with awk, cut, sed, etc.

Post by svnpenn » Tue, Jan 21, 2020, 10:44 PM UTC

0xff00ffee wrote: post #87

pizza234 wrote: post #57

Earlier quoted context omitted.

> I don't know of any easier equivalent of "awk '{ print $2 }'" for what it does. I'm not sure if you refer spefically to cut, but Perl has something similar and approximaly terse: > echo 'a b c' | perl -lane 'print $F[1]' Also, Perl can slice arrays, which is something that I really miss in Awk.

PERL is bloatware by comparison and less likely to be installed on distros than AWK. (e.g, embedded or slim distros. that's why you rarely see nonstandard /bin execs in shell scripts).

Post by 3fe9a03ccd14ca5 » Tue, Jan 21, 2020, 11:02 PM UTC

bloopernova wrote: post #83

I use awk because there's an almost 100% chance that it's going to be installed on any unix system I can ssh into. I use awk because I like to visually refine my output incrementally. By combining awk with multiple other basic unix commands and pipes, I can get the data that I want out of the data I have. I'm not writing unit tests or perfect code, I'm using rough tools to do a quick one-off job. For instance, "mail…

Post by emmelaich » Tue, Jan 21, 2020, 11:03 PM UTC

jcims wrote: post #120

bloopernova wrote: post #83

I use awk because there's an almost 100% chance that it's going to be installed on any unix system I can ssh into. I use awk because I like to visually refine my output incrementally. By combining awk with multiple other basic unix commands and pipes, I can get the data that I want out of the data I have. I'm not writing unit tests or perfect code, I'm using rough tools to do a quick one-off job. For instance, "mail…

I know you’re not asking for awk protips but you can prefix the block with a match condition for processing. ... | grep foo | awk ‘{print $6}’ | ... becomes ... | awk ‘/foo/{print $6}’ | ... If you start working this into your awk habits you’ll find delightful little edge cases that you can handle with other expressions before the block (you can, for example, match specific fields).

   $ echo fish | awk  '/\bfish\b/' 

   $ echo fish | perl -ne  '/\bfish\b/ && print' 

Post by _slyo » Tue, Jan 21, 2020, 11:05 PM UTC

geocrasher wrote: post #63

My favorite Awk reference is this: https://ferd.ca/awk-in-20-minutes.html Also, a handy trick is to combine awk and cut. For example I had a log line that had a variable amount of columns just in one field, but immediately after the field was a comma. I cut based on the comma: cut -d, -f1,2 and then awk'd the last column: cut -d, -f1,2 | awk '{ print $2" "$5" "$NF }' So, sometimes awk and cut can help each other.