Live data from Hacker News

Awk in 20 Minutes (2015)

ferd.ca

31–40 of 128 posts

Re: Awk in 20 Minutes (2015)

#31

I learned awk from the chapter on filters in the pamphlet sized book 'unix programming environment' by Kernighan and Pike you can buy for $8

I've heard many good things about that book, which is why I bought it several months ago. From the bits I have read, I like it. But I have not yet finished it, partly because it is not pamphlet sized. It is over 300 pages. But I look forward to the chapter on Filters.

Re: Awk in 20 Minutes (2015)

#33
post #10

So, I don't use awk right now. This answers "how to awk" more than "why use awk" for me (understandable given the 20 min claim). Does anyone have concrete, practical examples of use cases where awk made your life much easier?

I wrote my answer to this a few years ago:

https://www.bignerdranch.com/blog/a-crash-course-in-awk/

A tl;dr for the following tl;dr: it's great for quick-and-dirty processing of logs.

The tl;dr is that AWK is simple language that lets you use a line-oriented event driven programming model to process text. This abstraction is simple enough to be readable and maintainable, but powerful enough to parse and process text with line-oriented structure.

The example linked elsewhere is worth studying, because it truly is one of the prettiest pieces of programming I've seen in a long while:

https://c2.com/doc/expense/

If you don't know anything about AWK, here's all you need to know to grok this:

1. An AWK program is a list of [event] { code } pairs. All pairs are run in sequence on each line of input; if the "event" evaluates to true or is a matching regex, the matching code runs.

2. { code } on its own runs unconditionally.

3. The input is automatically whitespace delimited into fields; $1 refers to field 1, $2 to field 2, etc

4. NF is a special value that yields the number of fields

5. Assigning to fields replaces the text in those fields.

6. ($1+0) != 0 implicitly converts $1 to an integer; if it fails, the value is 0.

There is a lot of implicit loose typing in the language, and usage of undefined variables is idiomatic. Functions aren't easy to use, either. So it's not well-suited for programming in the large, or even the medium. But for the scale of programming seen in that link, it's truly a wonderful and simple power tool.

Re: Awk in 20 Minutes (2015)

#34
post #10

So, I don't use awk right now. This answers "how to awk" more than "why use awk" for me (understandable given the 20 min claim). Does anyone have concrete, practical examples of use cases where awk made your life much easier?

I use awk when I need something a bit more powerful than "grep"/"cut", but don't want to pull out the big guns with Perl. For example, recently I needed to print out certain fields of an output, but only for a given subset. So I used Awk to create a simple state machine (enable when I see the start of the subset, disable at the end), and print the fields of interest.

The grep/cut replacement is exactly what I use awk the most for. I use it for other things too but this stands out. awk just fixes the problem when you need to have more than one field from the input and maybe a different string between them. I guess this can be done with cut too, but for me it's simpler this way.

Also I use awk often in scripts together with fzf to interactively switch contexts e.g. in cloud provider CLIs.

Re: Awk in 20 Minutes (2015)

#35

I learned perl in college in the 5.6 days, and did a lot of text processing with it for a time. At some point, I bit the bullet and learned awk, and I've mostly abandoned perl as a result. Why? awk is small enough that it fits in my head, or at least the bits I need every couple of months do. And if I forget, it only takes 20 minutes to put them back in. perl, by contrast, is far too large to fit in my head and comes…

Perl is Awk on steroids :)

Re: Awk in 20 Minutes (2015)

#37
post #19
post #15

Earlier quoted context omitted.

LWSP=Linear White Space "Linear white space is: any number of spaces or horizontal tabs, and also newline (CRLF) if it is followed by at least one space or horizontal tab." [1] I don't know what LWSP gobbling is, though. Google didn't help. [1] https://stackoverflow.com/questions/21072713/what-exactly-is...

It means a contiguous collection of arbitrary white space characters are considered to be a single delimiter. With the parent example it means that column #2 could have any of number whitespaces between it and column #1.

yes. whereas cut, considers ' ' (two spaces) as an instance of a blank column separated by the -d' ' space instance. Its like ,, in CSV being a blank column.

Re: Awk in 20 Minutes (2015)

#38
post #36

Common lines between 2 files awk 'NR==FNR{a[$0]; next} $0 in a' colors_1.txt colors_2.txt http://archive.vn/mmd80

I've recently modified and published that tutorial as an ebook [1], which can be read from the github repo or downloaded as PDF (currently free). I am also updating the book to include exercises, other minor improvements as well as epub version - expected release by next weekend.

[1] https://github.com/learnbyexample/learn_gnuawk

Re: Awk in 20 Minutes (2015)

#39
post #11
post #10

So, I don't use awk right now. This answers "how to awk" more than "why use awk" for me (understandable given the 20 min claim). Does anyone have concrete, practical examples of use cases where awk made your life much easier?

Awk '{ print $2 }' does LWSP gobbling.. cut -d' ' -f2 doesn't and this difference alone makes awk useful to me on a daily basis. Awk has a hash like perl which is very efficient. I use an awk expression to print uniq as they come in counted through the hash insert on new instead of uniq which prints at end. Awk count unique over 300,000,000 ips was as fast as perl and python and smaller memory footprint

> Awk '{ print $2 }' does LWSP gobbling

That's like answering "why use Haskell?" with "Because its monads are monoids in the category of endofunctors"

Re: Awk in 20 Minutes (2015)

#40
post #11
post #10

So, I don't use awk right now. This answers "how to awk" more than "why use awk" for me (understandable given the 20 min claim). Does anyone have concrete, practical examples of use cases where awk made your life much easier?

Awk '{ print $2 }' does LWSP gobbling.. cut -d' ' -f2 doesn't and this difference alone makes awk useful to me on a daily basis. Awk has a hash like perl which is very efficient. I use an awk expression to print uniq as they come in counted through the hash insert on new instead of uniq which prints at end. Awk count unique over 300,000,000 ips was as fast as perl and python and smaller memory footprint

If you just use awk for `{ print $2 }` then I would still prefer `tr -s ' ' | cut -d ' ' -f2`, since both are part of coreutils and with `awk` you would add a additional dependency to your script.
Post reply on HN