Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Awk in 20 Minutes (2015)

ferd.ca

Post reply on HN
Print view View original

41–50 of 128 posts

Re: Awk in 20 Minutes (2015)

#41

Post by arexxbifs »

ggm wrote: post #11
bcyn wrote: post #10

So, I don't use awk right now. This answers "how to awk" more than "why use awk" for me (understandable given the 20 min claim). Does anyone have concrete, practical examples of use cases where awk made your life much easier?

Awk '{ print $2 }' does LWSP gobbling.. cut -d' ' -f2 doesn't and this difference alone makes awk useful to me on a daily basis. Awk has a hash like perl which is very efficient. I use an awk expression to print uniq as they come in counted through the hash insert on new instead of uniq which prints at end. Awk count unique over 300,000,000 ips was as fast as perl and python and smaller memory footprint

tr will handle the whitespace for you and is generally faster[0] (but awk is faster to type and reads better).

[0] https://datagubbe.se/cutvawk/

Re: Awk in 20 Minutes (2015)

#42

Post by asicsp »

chme wrote: post #40
ggm wrote: post #11

Earlier quoted context omitted.

Awk '{ print $2 }' does LWSP gobbling.. cut -d' ' -f2 doesn't and this difference alone makes awk useful to me on a daily basis. Awk has a hash like perl which is very efficient. I use an awk expression to print uniq as they come in counted through the hash insert on new instead of uniq which prints at end. Awk count unique over 300,000,000 ips was as fast as perl and python and smaller memory footprint

If you just use awk for `{ print $2 }` then I would still prefer `tr -s ' ' | cut -d ' ' -f2`, since both are part of coreutils and with `awk` you would add a additional dependency to your script.

that is not equivalent because awk will remove trailing/leading space/tab/newlines (newlines come into play with a different record separator)

whereas, tr will still leave a trailing/leading space

for example:

    $ echo '   a    b  c   ' | tr -s ' ' | cut -d ' ' -f2
    a
    $ echo '   a    b  c   ' | awk '{print $2}'
    b
and by default awk splits on space/tab/newlines, whereas in cut example above, you get only space as delimiter

cut has its uses and will be faster than awk, but it depends on the problem being solved

Re: Awk in 20 Minutes (2015)

#43

Post by ggm »

skrebbel wrote: post #39
ggm wrote: post #11

Earlier quoted context omitted.

Awk '{ print $2 }' does LWSP gobbling.. cut -d' ' -f2 doesn't and this difference alone makes awk useful to me on a daily basis. Awk has a hash like perl which is very efficient. I use an awk expression to print uniq as they come in counted through the hash insert on new instead of uniq which prints at end. Awk count unique over 300,000,000 ips was as fast as perl and python and smaller memory footprint

> Awk '{ print $2 }' does LWSP gobbling That's like answering "why use Haskell?" with "Because its monads are monoids in the category of endofunctors"

This made me very happy, since it goes to the 'you can either use awk or explain awk but not both' problem which is definitional in monads in Haskell. I thought LWSP was a thing. I forgot not everything is a thing. And gobbling is maybe very jargon ish. Python sys.stdin.rstrip().split() comes to mind.

Re: Awk in 20 Minutes (2015)

#45

Post by mkl »

known wrote: post #36

Common lines between 2 files awk 'NR==FNR{a[$0]; next} $0 in a' colors_1.txt colors_2.txt http://archive.vn/mmd80

The link doesn't load for me, but that seems pretty unreadable. I'd probably sort, merge, and print duplicates:

  sort -m

Re: Awk in 20 Minutes (2015)

#46

Post by j_z_reeves »

nice, I finally took the time to read the man pages for awk. And whipped out a script to count the number of errors occurred for a particular day for a postgres log file.

   cat logfile | awk '/ERROR:/ {counts[$1] = counts[$1] + 1}; END { for (day in counts) print day " : " counts[day]}' | sort
I just needed to know how awk programs are structured, the rest is just simple programming!

EDIT: I'm not sure if it's actually correct however...

Re: Awk in 20 Minutes (2015)

#47

Post by Marcus316 »

I love using awk. It was fairly easy to pick up, and it slides into my command line workflow pretty well.

That said, there's an amazing amount you can do with it, if you really try. Someone once joked that I should try building an IRC bot in AWK, so I did: https://github.com/Marcus316/rufus

There's no pactical reason for it, but it was fun to play with the idea.

Re: Awk in 20 Minutes (2015)

#48

Post by asicsp »

mkl wrote: post #45
known wrote: post #36

Common lines between 2 files awk 'NR==FNR{a[$0]; next} $0 in a' colors_1.txt colors_2.txt http://archive.vn/mmd80

The link doesn't load for me, but that seems pretty unreadable. I'd probably sort, merge, and print duplicates: sort -m

you can use comm as well

    # common lines 
    comm -12
regarding readability, it is the same with any new tool or programming language, you'd need to be familiar with its syntax and idioms, someone not familiar with command line and sort/uniq commands will find your solution as alien as well

Re: Awk in 20 Minutes (2015)

#49

Post by Noumenon72 »

yaktubi wrote: post #7
xwdv wrote: post #6

Every time I set out to use awk for some one time task I struggle to express what I need and end up saying fuck it and fire up vim and do exactly what I want with some quick macros. Am I the only one?

I usually say duck it and dust off my shiny Perl necklace

I usually persevere and get it done in awk but then find the next time I need awk that I don't remember any of it and can barely understand my own examples.

Re: Awk in 20 Minutes (2015)

#50

Post by em500 »

chme wrote: post #40
ggm wrote: post #11

Earlier quoted context omitted.

Awk '{ print $2 }' does LWSP gobbling.. cut -d' ' -f2 doesn't and this difference alone makes awk useful to me on a daily basis. Awk has a hash like perl which is very efficient. I use an awk expression to print uniq as they come in counted through the hash insert on new instead of uniq which prints at end. Awk count unique over 300,000,000 ips was as fast as perl and python and smaller memory footprint

If you just use awk for `{ print $2 }` then I would still prefer `tr -s ' ' | cut -d ' ' -f2`, since both are part of coreutils and with `awk` you would add a additional dependency to your script.

Why would being part of coreutils matter? Awk is part of POSIX, just like tr and cut. Even in very constraint environments you can count on it via busybox.
Post reply on HN