Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CLI text processing with GNU awk

learnbyexample.github.io

Post reply on HN
Print view View original

71–80 of 136 posts

Re: CLI text processing with GNU awk

#71

Post by hereonout2 »

ajross wrote: post #43
e63f67dd-065b wrote: post #34

Perhaps my old sysadmin hat is showing through, but I don’t quite see what the advantage of awk is over just writing the same thing in perl. I’ve seen my fair share of horrendous shell scripts from junior sysadmins, and every time I think to myself “the text processing portion would be so much cleaner in Perl”.

There aren't any technical advantages, no. Perl's features are a proper superset of awk (by design!). What's happened is that Kids Today (tm) never learned perl. So they're discovering awk as someone new to the idea of stream processing. And awk was a great idea for that, and it represented a genuine innovation worth emulating. In the late 1970's. Then of course perl did emulate and surpass it. But then got forgotten…

Perl came out in the late 80s and by the mid 2000's was really on its way out? I hired for my last perl position in around 2006.

Just saying, your definition of "kids today" could well include a decent portion of developers under 45 years old. Referring to this cohort repeatedly as "kids" is also a little cringe.

Re: CLI text processing with GNU awk

#72

Post by TeMPOraL »

pletnes wrote: post #63
ajross wrote: post #53

Earlier quoted context omitted.

But awk is never used alone. You don't solve whole problems with awk, you squish it into a script with a bunch of other junk. My point is that you're making an apples-to-oranges comparison. Sure, "awk" isn't the problem, but "bash" is, and bash is undeniably a more error-prone language than perl. You surely agree with that much, right? And if you disallow "bash" for security reasons, where does that leave "awk" in th…

You’re right. But the alternative might be bash + perl or just bash. Or none of them. Perl is anyway the first one to go.

Is it completely gone, or rather just for you, blocked by sysadmins who know Perl is the magic pixie dust for total control, and want to keep it for themselves?

In Windows-land, compare how PowerShell access may be restricted, and you won't be allowed to run macros in Office, all while your computer is "managed" by a horrible hodge-podge of PowerShell and VBA scripts that make Perl code look like high literature.

Re: CLI text processing with GNU awk

#73

Post by mistercheph »

swores wrote: post #21
dotancohen wrote: post #15

Earlier quoted context omitted.

ChatGPT is a great time saver for those who already know how to use awk. But it should not be used by those who are unfamiliar. Just an example, I saw someone come up with a great awk line to change some text in a nested directory. He then pasted into bash. Only once the server went down did anybody realize that he forgot to cd into the proper directory and he wiped out not only the server config but also all the use…

That's not really a ChatGPT issue, people pasting in slightly wrong commands (or right commands in the wrong folder) is a tale as old as time - well, as old as linux at least. Short of saying that nobody who's already an expert should ever touch a CLI, the lesson from that story is "be as careful as possible, then be more careful, and also have backups of everything" not "don't use a LLM to help".

What “care” do you suggesst that someone pasting in a script they don’t understand should take?

Re: CLI text processing with GNU awk

#76

Post by ajross »

hereonout2 wrote: post #71
ajross wrote: post #43

Earlier quoted context omitted.

There aren't any technical advantages, no. Perl's features are a proper superset of awk (by design!). What's happened is that Kids Today (tm) never learned perl. So they're discovering awk as someone new to the idea of stream processing. And awk was a great idea for that, and it represented a genuine innovation worth emulating. In the late 1970's. Then of course perl did emulate and surpass it. But then got forgotten…

Perl came out in the late 80s and by the mid 2000's was really on its way out? I hired for my last perl position in around 2006. Just saying, your definition of "kids today" could well include a decent portion of developers under 45 years old. Referring to this cohort repeatedly as "kids" is also a little cringe.

Did you really just reply to a comment that used the phrase "Kids Today (tm)" and try to interpret it as a genuine insult? The inability of this community to understand straightforward humor amazes me. Dude, it was a joke. And yes, I was calling mid-career professionals "kids". Deliberately. Because I'm old. And it's funny.

Re: CLI text processing with GNU awk

#77

Post by swores »

mistercheph wrote: post #73
swores wrote: post #21

Earlier quoted context omitted.

That's not really a ChatGPT issue, people pasting in slightly wrong commands (or right commands in the wrong folder) is a tale as old as time - well, as old as linux at least. Short of saying that nobody who's already an expert should ever touch a CLI, the lesson from that story is "be as careful as possible, then be more careful, and also have backups of everything" not "don't use a LLM to help".

What “care” do you suggesst that someone pasting in a script they don’t understand should take?

I'm far from an expert so you should probably ask someone other than me. But my two cents would be not to paste any code until you have understood it, or unless it's written by a source you trust, or alternatively only paste it somewhere you don't care - when I'm playing around testing stuff I might not fully understand on a linux server I do it on a VPS that's unimportant to me, and that if I mess it up I can very easily restore it back to a clean OS install and I have a bash script ready to reinstall all software I want & all the profile customisations etc.

Re: CLI text processing with GNU awk

#78

Post by ra1231963 »

Never learned awk or committed esoteric cli incantations to memory. Don’t get me wrong, I can get around on the cli, but sed, awk, etc just didn’t seem like a good cost/benefit investment. I’m also not a sysadmin.

Thankfully I waited long enough and LLMs can write them for me better than I ever could.

Re: CLI text processing with GNU awk

#79

Post by Anthony-G »

chasil wrote: post #30
ymgch wrote: post #7

What is better? Starting with awk or sed?

A sed binary is usually much smaller than an awk binary, either POSIX or GNU. The memory footprint of sed will be much more compact. However, sed has grown out of the command language used by the tty editors, and is more difficult to program (although it is Turing-complete). The awk language implements much of the syntax of C, and it is not difficult to write a very slow and inefficient script. This inefficiency is h…

For those who care about copyright, that URL is not from O’Reilly; it’s a copy of a book-set that O’Reilly used to distribute via CD-ROM – with a nice user interface that used web technologies (even included a search feature). O’Reilly could make it available for free – as they’ve done so for other books such as Apache Security¹ or Using Samba² – but they still (as is their right) expect you to pay for the sed & awk book³.

¹ https://blog.ivanristic.com/2015/02/apache-security-ten-year...

² https://www.oreilly.com/openbook/samba/book/

³ https://www.oreilly.com/library/view/sed-awk/1565922255/

Re: CLI text processing with GNU awk

#80

Post by martincmartin »

nologic01 wrote: post #61

awk one-liners are a slam dunk. The tough question whether to invest in more complex awk programming. Invariably some processing task requires more complex logic and awk provides that, but in the terse and arcane ways of early computing. Yet reaching for a modern alternative is also an overhead, may not be particularly intuitive either (hello pandas) and may even have performance issues...

For me, the big problem is libraries. Even a personal file of common functions doesn't seem that well supported, and there just doesn't seem to be a way to get third party libraries.

When I start needing helper functions and splitting it into multiple lines is usually when I reach for Python instead. And then sigh, because my program will be 2 to 3 times bigger. Ruby is a great awk replacement, but unless other people at your job know it, you can't expect others to maintain it.

Post reply on HN