Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CLI text processing with GNU awk

learnbyexample.github.io

Post reply on HN
Print view View original

41–50 of 136 posts

Re: CLI text processing with GNU awk

#41

Post by asicsp »

e63f67dd-065b wrote: post #34

Perhaps my old sysadmin hat is showing through, but I don’t quite see what the advantage of awk is over just writing the same thing in perl. I’ve seen my fair share of horrendous shell scripts from junior sysadmins, and every time I think to myself “the text processing portion would be so much cleaner in Perl”.

If you are comparing Awk vs Perl for scripts, I'd prefer Perl (or Python).

This post is about short one-liners for ad hoc use cases. I prefer sed/awk over Perl for such cases. Though, if you already know Perl, you could continue using it instead of having to learn more tools.

Re: CLI text processing with GNU awk

#42

Post by ajross »

pletnes wrote: post #36
e63f67dd-065b wrote: post #34

Perhaps my old sysadmin hat is showing through, but I don’t quite see what the advantage of awk is over just writing the same thing in perl. I’ve seen my fair share of horrendous shell scripts from junior sysadmins, and every time I think to myself “the text processing portion would be so much cleaner in Perl”.

Remove perl, have less security issues. Some scanning tools flag it, too. Awk is found in more places, in my experience.

Sorry, but that's ridiculous. Any general purpose programming language is a vector for bugs and security problems, but come on: you're genuinely trying to say that a kludge of bash+sed+awk is objectively more "secure" than a single perl script to solve the same problem?

Re: CLI text processing with GNU awk

#43

Post by ajross »

e63f67dd-065b wrote: post #34

Perhaps my old sysadmin hat is showing through, but I don’t quite see what the advantage of awk is over just writing the same thing in perl. I’ve seen my fair share of horrendous shell scripts from junior sysadmins, and every time I think to myself “the text processing portion would be so much cleaner in Perl”.

There aren't any technical advantages, no. Perl's features are a proper superset of awk (by design!).

What's happened is that Kids Today (tm) never learned perl. So they're discovering awk as someone new to the idea of stream processing. And awk was a great idea for that, and it represented a genuine innovation worth emulating.

In the late 1970's. Then of course perl did emulate and surpass it. But then got forgotten. So kids are discovering awk instead. It's a little cringe, really.

Re: CLI text processing with GNU awk

#45

Post by thesuperbigfrog »

delta_p_delta_x wrote: post #14
qalmakka wrote: post #13

Awk is fine and dandy but, like wity Sed, I think that it's almost always replaceable with Perl which is way nicer to use, and ubiquitous. Every OS (except Windows) I laid my hands on in the last 15 years has had Perl installed in either its default install or pulled in as a dependency almost immediately (a LOT of stuff depends on Perl in any Unix system). This is, unless you are running on an embedded environment, b…

> except Windows On Windows, you use PowerShell.

>> On Windows, you use PowerShell.

If you use Git for Windows (https://gitforwindows.org/), it includes Perl.

Or you could install Strawberry Perl which is made for Windows: https://strawberryperl.com/

Re: CLI text processing with GNU awk

#46

Post by macintux »

cb321 wrote: post #38

Of possible interest - instead of making a whole new programming language like awk, you can also just systematize generating code for an existing one with a command-line harness. This can even stay terse & keep a fairly fast edit-test turnaround in a fully statically typed language like Nim: https://github.com/c-blake/bu/blob/main/doc/rp.md

Whole new language? Awk is 45 years old.

Re: CLI text processing with GNU awk

#47

Post by Bjartr »

asicsp wrote: post #26
swores wrote: post #20

Earlier quoted context omitted.

I'm curious to know how many people you get paying for something like your "Magical one-liners", and also whether you've ever experimented with a "choose to pay after" model? I ask because it's the kind of thing that I can imagine finding useful enough to pay $5 (or $15) for, but I can also imagine it being something that contains nothing I don't already have saved in my personal "one liners" file, so I'm not really…

I use Gumroad/Leanpub to sell my ebooks. As far as I know, they don't support the "choose to pay after" model. You can see the number of paid sales for the bundles under the "I want this!" button. When the price is 0, it shows the total of both paid/free users. I started selling ebooks about 5 years back. Where I live, my monthly living cost is just $150. While the first two years of sales were just about enough to c…

> Where I live, my monthly living cost is just $150.

That's quite low! Mind if I ask where in the world that is?

Re: CLI text processing with GNU awk

#48

Post by macintux »

ajross wrote: post #43
e63f67dd-065b wrote: post #34

Perhaps my old sysadmin hat is showing through, but I don’t quite see what the advantage of awk is over just writing the same thing in perl. I’ve seen my fair share of horrendous shell scripts from junior sysadmins, and every time I think to myself “the text processing portion would be so much cleaner in Perl”.

There aren't any technical advantages, no. Perl's features are a proper superset of awk (by design!). What's happened is that Kids Today (tm) never learned perl. So they're discovering awk as someone new to the idea of stream processing. And awk was a great idea for that, and it represented a genuine innovation worth emulating. In the late 1970's. Then of course perl did emulate and surpass it. But then got forgotten…

For simple uses cases, I find awk simpler than Perl. I love Perl, have written tens of thousands of lines, but on the CLI I prefer awk. I’m sorry I “cringe” you.

Re: CLI text processing with GNU awk

#49

Post by cb321 »

macintux wrote: post #46
cb321 wrote: post #38

Of possible interest - instead of making a whole new programming language like awk, you can also just systematize generating code for an existing one with a command-line harness. This can even stay terse & keep a fairly fast edit-test turnaround in a fully statically typed language like Nim: https://github.com/c-blake/bu/blob/main/doc/rp.md

Whole new language? Awk is 45 years old.

I agree that "writing/learning a different" (what I meant) is more clear wording than "making a whole new".

EDIT: and it is a fair counterpoint that any command with options is, in some sense, also a different language one must learn. Learning API calls is also a different language (at least nouns & verbs if not syntax). But that is all partly the point. awk did/does a programming language with different syntax where other alternatives might be enough.

Re: CLI text processing with GNU awk

#50

Post by coliveira »

ajross wrote: post #42
pletnes wrote: post #36

Earlier quoted context omitted.

Remove perl, have less security issues. Some scanning tools flag it, too. Awk is found in more places, in my experience.

Sorry, but that's ridiculous. Any general purpose programming language is a vector for bugs and security problems, but come on: you're genuinely trying to say that a kludge of bash+sed+awk is objectively more "secure" than a single perl script to solve the same problem?

In the case of awk, actually yes, it is safer. The reason is that awk is a very limited language. It has only enough functionality to provide text matching and substitution. It is very difficult to use awk to do anything of high security risk, compared to a language like perl.
Post reply on HN