Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Learn Awk? (2016)

blog.jpalardy.com

Post reply on HN
Print view View original

51–60 of 246 posts

Re: Why Learn Awk? (2016)

#51

Post by masklinn »

cauthon wrote: post #34
jerf wrote: post #9

Because for some bizarre reason, "cut" doesn't ship with any decent column selection logic that is the equivalent of awk's $1, $2, etc., even in 2020. That's like 90% of my use of awk right there. I don't know of any easier equivalent of "awk '{ print $2 }'" for what it does. Posted partially so the Internet Correction Squad can froth at the mouth and set me straight, because I'd love to be showed to be wrong here.

> I don't know of any easier equivalent of "awk '{ print $2 }'" for what it does. Does `cut -f2` not work? My complaint with cut is that you can't reorder columns (e.g. `cut -f3,2` ) Awk is really great for general text munging, more than just column extraction, highly recommend picking up some of the basics Edit to agree with the commenters below me: If the file isn't delimited with a single character, then cut alon…

> Does `cut -f2` not work?

Most utilities don't use a tab character as separator, and that's what cut operates to by default. Can't cut on whitespace in general, which is what's actually useful, and what awk does.

Only way to get cut to work is to add a tr inbetween, which is a waste of time when awk just does the right thing out of the box.

Re: Why Learn Awk? (2016)

#52

Post by linsomniac »

samatman wrote: post #25

> Imagine programming without regular expressions. I live in this future and it's beautiful. Steps to programming without regular expressions: 1) find a PEG library for your language of choice There is no step 2.

TL;DR: PEG looks to me like simplified lex+yacc

For those not familiar with PEG, like me, here: https://en.wikipedia.org/wiki/Parsing_expression_grammar

I found that fairly abstract, a Python example provided more concrete examples: https://github.com/erikrose/parsimonious

Re: Why Learn Awk? (2016)

#53

Post by jolmg »

rhombocombus wrote: post #20
rectang wrote: post #10

Awk, like shell, is a fraught programming environment, full of silent failure and hidden gotchas. Even for one-liners. I use shell because I have to, not because I like it. I dread maintaining shell scripts which have a bunch of awk and sed in them. The Unix ideal of small single-purpose tools and text processing is separable from these old warhorses.

While I agree that the silent failures and "opaqueness" can be off-putting, once you understand the tool and how to implement it in your workflow it is wonderfully efficient. Aside from being syntactically terse I haven't found any compelling reasons not to use it.

> Aside from being syntactically terse I haven't found any compelling reasons not to use it.

For me, that's THE reason to use it. It's terseness is what allows it to be efficient enough to be used primarily interactively.

Re: Why Learn Awk? (2016)

#54

Post by Twisol »

umvi wrote: post #23
rb808 wrote: post #16

Earlier quoted context omitted.

scripts are the antithesis of modern software development. No Unit testing, No CI, no monitoring, no consistency, often no source control. Appeals to me as a hacker but only my scripts are intelligible - which is a bad sign. :)

I'm not actually convinced unit testing is all that valuable, unless the unit under test has a very clear input -> output transformation (like algorithms, string utilities, etc). If it doesn't (and most units don't), unit tests just encumber you.

The value of unit tests when creating software is perhaps debatable, but I find their greatest value to be in maintaining software. If you lock in your expectations of a software component, when it's time to make changes (due to shifting requirements or what not), you can add tests capturing your intent and make sure that you aren't breaking existing expectations. Or at least, that you know which expectations your change will break.

Re: Why Learn Awk? (2016)

#55

Post by tejtm »

iagovar wrote: post #38
samatman wrote: post #25

> Imagine programming without regular expressions. I live in this future and it's beautiful. Steps to programming without regular expressions: 1) find a PEG library for your language of choice There is no step 2.

Sorry for asking such dub question, but what's a PEG library?

Parsing expression grammar;

It is a recursive decent parser with the tiny tweak that productions are ordered (not a set) and short circuit.

A 90's language you did not have to imagine that saved you from regex in this way was[is] REBOL with it built in `parse` dsl.

examples here http://www.rebol.org/search.r?find=parse&form=yes

[0]https://en.wikipedia.org/wiki/Parsing_expression_grammar [1]http://www.rebol.com/

Re: Why Learn Awk? (2016)

#56

Post by cauthon »

masklinn wrote: post #51
cauthon wrote: post #34

Earlier quoted context omitted.

> I don't know of any easier equivalent of "awk '{ print $2 }'" for what it does. Does `cut -f2` not work? My complaint with cut is that you can't reorder columns (e.g. `cut -f3,2` ) Awk is really great for general text munging, more than just column extraction, highly recommend picking up some of the basics Edit to agree with the commenters below me: If the file isn't delimited with a single character, then cut alon…

> Does `cut -f2` not work? Most utilities don't use a tab character as separator, and that's what cut operates to by default. Can't cut on whitespace in general, which is what's actually useful, and what awk does. Only way to get cut to work is to add a tr inbetween, which is a waste of time when awk just does the right thing out of the box.

> which is a waste of time when awk just does the right thing out of the box.

Agree in general. Only exception I'd make to this is when you're selecting a range of columns, as someone else mentioned elsewhere in the thread. I typically find (for example) `| sed -e 's/ \+/\t/g' | cut -f 1,3-10,14-17` to be both easier to type and easier to debug than typing out all the columns explicitly in an awk statement.

Re: Why Learn Awk? (2016)

#57

Post by pizza234 »

jerf wrote: post #9

Because for some bizarre reason, "cut" doesn't ship with any decent column selection logic that is the equivalent of awk's $1, $2, etc., even in 2020. That's like 90% of my use of awk right there. I don't know of any easier equivalent of "awk '{ print $2 }'" for what it does. Posted partially so the Internet Correction Squad can froth at the mouth and set me straight, because I'd love to be showed to be wrong here.

> I don't know of any easier equivalent of "awk '{ print $2 }'" for what it does.

I'm not sure if you refer spefically to cut, but Perl has something similar and approximaly terse:

> echo 'a b c' | perl -lane 'print $F[1]'

Also, Perl can slice arrays, which is something that I really miss in Awk.

Re: Why Learn Awk? (2016)

#58

Post by just_myles »

shpeedy wrote: post #31

Skip learning of sed and awk and jump straight to perl instead. $ perl --help ... -F/pattern/ split() pattern for -a switch (//'s are optional) -l[octal] enable line ending processing, specifies line terminator -a autosplit mode with -n or -p (splits $_ into @F) -n assume "while ( ) { ... }" loop around program -p assume loop like -n but print line also, like sed -e program one line of program (several -e's allowed,…

Because syntatically as a language/tool it is super easy to remember. Writing one liners with awk feels more intuitive to me.

Awk example:

ls -l | awk '{print $9, $5}' or ls -lh | awk '{print $9, $5}'

Seems a whole lot simpler. To me. I find if you have to write exhaustive shell scripts then maybe you can look for something more verbose like Perl, I guess.

Re: Why Learn Awk? (2016)

#59

Post by b215826 »

cauthon wrote: post #45
ColinWright wrote: post #37

Earlier quoted context omitted.

I'm an expert on neither AWK nor cut , but AWK allows you to select a field, or the entire line, and then substrings within those. Select characters 3-6 (inclusive) in the second field: $ echo Testing LengthyString | awk '{print substr($2,3,4)}' > ngth If you want to select columns from the entire line, then: $ echo Testing LengthyString | awk '{print substr($0,3,4)}' > stin Is that what you meant?

I don't think so. I think they're referring to cut's ability to select an arbitrary range of columns, e.g. `cut -f 2-7` to select the 2nd through 7th columns, while awk requires you to explicitly enumerate all desired columns, i.e. `awk '{print $2, $3, $4, $5, $6, $7}'

> "while awk requires you to explicitly enumerate all desired columns"

Or you can use loops, e.g.,

  echo $(seq 100) | awk '{ for (i = 2; i

Re: Why Learn Awk? (2016)

#60

Post by PaulDavisThe1st »

msla wrote: post #36
sremani wrote: post #2

Honorary mention of Taco Bell Programming. (fits this genre). http://widgetsandshit.com/teddziuba/2010/10/taco-bell-progra... Someone ought to write - Zen and the art of Unix tools usage.

Simple, Composable Pieces is practically the whole ethos of both Unix and Functional Programmers, and they both converged on the same basic flow model: Pipelines with minimal side-effects. This naturally leads to a sequential concurrency, which seems to be easy for humans to reason about: Data flows through the pipeline, different parts of the pipeline can all be active at once, and nobody loses track. It doesn't sol…

Works until time plays a role in the computation you're doing.
Post reply on HN