Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Awk in 20 Minutes (2015)

ferd.ca

Post reply on HN
Print view View original

21–30 of 128 posts

Re: Awk in 20 Minutes (2015)

#21

Post by mauvehaus »

I learned perl in college in the 5.6 days, and did a lot of text processing with it for a time. At some point, I bit the bullet and learned awk, and I've mostly abandoned perl as a result.

Why? awk is small enough that it fits in my head, or at least the bits I need every couple of months do. And if I forget, it only takes 20 minutes to put them back in. perl, by contrast, is far too large to fit in my head and comes with an ecosystem that is much larger still.

I know I can use perl for what I use awk for, and when I've raised this point before, people have been quick to explain how to process input by lines, and conditionally do something. For basic stuff, the fact that that's basically all awk does[0] means there's so many fewer ways to do it wrong. I can't say the same of perl, even when I was more familiar with it.

caveat: awk, nawk, mawk, and gawk don't necessarily share the same set of corner cases, and you may not get error messages that make sense to you when you bump into one with an unfamiliar awk.

[0] I know, not really, and especially for gawk. I've written awk scripts a couple hundred lines in the past. It's true enough for the 20 minute version though.

Edited for clarity.

Re: Awk in 20 Minutes (2015)

#22

Post by gen220 »

mauvehaus wrote: post #20
gen220 wrote: post #16

Earlier quoted context omitted.

I use awk to write csv-parsers for my personal finance setup. They take transaction logs and convert them into a ledger format ( https://www.ledger-cli.org/ ). There's some pretty trivial if/else branches based on regexp's, which awk is pretty good at expressing. I'd normally write this sort of thing in python. The awk program is usually smaller than a comparable python program, but for me the main selling point is t…

I've used it for this too, but things tend to go south pretty quick if you have escaped or quoted commas. That, unfortunately, is where I usually break down and pull out python.

hehe, I hit that bump in the road too, but maybe I was too stubborn.

FWIW, you can get around it. I just took a peek because I couldn't remember how to do it off the top of my head.

It looks like I had to use

  gawk -F ',' -v FPAT='([^,]+)|("[^"/]+")'
to get the behavior you're looking for. Seems like I nabbed it from https://www.gnu.org/software/gawk/manual/html_node/Splitting...

Agreed that this is the sane point at which to pull out python.

Re: Awk in 20 Minutes (2015)

#24

Post by asicsp »

bcyn wrote: post #10

So, I don't use awk right now. This answers "how to awk" more than "why use awk" for me (understandable given the 20 min claim). Does anyone have concrete, practical examples of use cases where awk made your life much easier?

Here's some articles

* https://blog.jpalardy.com/posts/why-learn-awk/ (also discussed on HN: https://news.ycombinator.com/item?id=22108680)

* https://adamdrake.com/command-line-tools-can-be-235x-faster-...

Re: Awk in 20 Minutes (2015)

#25

Post by AceJohnny2 »

mauvehaus wrote: post #21

I learned perl in college in the 5.6 days, and did a lot of text processing with it for a time. At some point, I bit the bullet and learned awk, and I've mostly abandoned perl as a result. Why? awk is small enough that it fits in my head, or at least the bits I need every couple of months do. And if I forget, it only takes 20 minutes to put them back in. perl, by contrast, is far too large to fit in my head and comes…

For my & others' reference (because I keep having to look it up, just like parent):

    perl -ne "print $_;"
'-n' will run the expression over each line of the input. This is the Awk-like mode.

    perl -pe "s/foo/bar/"
'-p' will run the expression over each line of input, and print out the (possibly modified) line. This is the Sed-like mode.

Slightly more info available in `man perlrun` or https://perldoc.perl.org/perlrun.html

Re: Awk in 20 Minutes (2015)

#26

Post by asicsp »

bcyn wrote: post #13
na85 wrote: post #12

Earlier quoted context omitted.

I contribute to a niche game server project and used awk to generate c# class files from an org mode document that I wrote to specify them.

What about awk makes that easier than, say, a Python script?

depends upon the usecase, for relatively simple column filtering, substitution, etc, awk script would be briefer than python equivalent and most likely be faster as well

if it becomes lengthy (and again depends on features required), then Python is likely to have inbuilt/3rd-party libraries to make it easier to write and maintain

if it is a question of constructing command line one-liners and using it as a part of other cli tools, then awk wins easily

for example:

    awk -F'\\W+' -v OFS=, '{print $NF, $2}' input.txt
prints last column and second column, where non-word characters form the field separator and comma is used as output field separator

Re: Awk in 20 Minutes (2015)

#27

Post by AceJohnny2 »

bcyn wrote: post #10

So, I don't use awk right now. This answers "how to awk" more than "why use awk" for me (understandable given the 20 min claim). Does anyone have concrete, practical examples of use cases where awk made your life much easier?

I use awk when I need something a bit more powerful than "grep"/"cut", but don't want to pull out the big guns with Perl.

For example, recently I needed to print out certain fields of an output, but only for a given subset. So I used Awk to create a simple state machine (enable when I see the start of the subset, disable at the end), and print the fields of interest.

Re: Awk in 20 Minutes (2015)

#28

Post by AceJohnny2 »

AceJohnny2 wrote: post #27
bcyn wrote: post #10

So, I don't use awk right now. This answers "how to awk" more than "why use awk" for me (understandable given the 20 min claim). Does anyone have concrete, practical examples of use cases where awk made your life much easier?

I use awk when I need something a bit more powerful than "grep"/"cut", but don't want to pull out the big guns with Perl. For example, recently I needed to print out certain fields of an output, but only for a given subset. So I used Awk to create a simple state machine (enable when I see the start of the subset, disable at the end), and print the fields of interest.

Or you could use it to solve the Towers of Hanoi game ;)

https://rosettacode.org/wiki/Towers_of_Hanoi#AWK

Re: Awk in 20 Minutes (2015)

#29

Post by dwoot »

geocrasher wrote: post #2

Whenever somebody says "Oh, I don't know how to use awk", this is the link I send them. It's easily the most useful tutorial for awk on the 'net. There's a lot more you can do with awk, but this site shows you the top percentage in a very short amount of time.

It's crazy how this reappeared on HN only hours after I went searching for awk on Hacker News. Fred Hebert is so well known in the Erlang and Elixir communities.

I have just coined this idea the "gateway drug" approach to learning new tools. We should strive to find those introductions that are small enough that someone can digest without a massive upfront time investment to get them past the front door :)

Re: Awk in 20 Minutes (2015)

#30

Post by daenz »

Very cool and I'm grateful for Awk to be presented in such a friendly way.

However, at some point, it makes more sense to write your Awk script in something like Python, and my intuition says that that time is shortly after starting a basic Awk script. Using a real programming language is almost always the way to go with code that will become more complex over time (almost all code) and code you have to share with a team (learning curve).

Post reply on HN