Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Awk: The Power and Promise of a 40-Year-Old Language

fosslife.org

Post reply on HN
Print view View original

111–120 of 122 posts

Re: Awk: The Power and Promise of a 40-Year-Old Language

#111

Post by teknopaul »

selfhoster11 wrote: post #71
dunefox wrote: post #37

Earlier quoted context omitted.

Ehh. Until the 'job' gets extended and then your simple tool makes it exponentially more complex and you have to rewrite it with the more powerful tool.

This is fine. I often "prototype" my automations as shell scripts, to explore what I actually want the tool to handle. Once it gets longer than 20 or so lines, it's time to move to a better language, but I don't mind rewriting. This is a chance to add error handling, config, proper arguments, built-in help texts and whatever else.

I started to add error handling to my shell scripts and often never rewrite them. Defo agree with the sentiment that you should always be happy (and able) to rewrite a shell scripts, dont let its scope creep. I don't mind long(ish) shell scripts as long as the program flow is fairly linear. Too many function calls is the smell that makes me rewrite.

Re: Awk: The Power and Promise of a 40-Year-Old Language

#112

Post by u801e »

abecedarius wrote: post #80
michaelcampbell wrote: post #54

Earlier quoted context omitted.

I'm curious what linux distros don't have either some version of perl or python. I like awk, mind, but this is not necessarily (IME) a good argument for it.

A nice thing about awk vs. Perl/Python: there's a small focused set of things to learn. Once you learn them you're done. This suggests an opening for a Perl/Python intro focused on the exact same tasks, admittedly. That seems more realistic for Perl -- unless there's someone who writes Python one-liners at the shell?

perl actually has a one-liner way of invocation that's modeled after awk.

For example, to print the first field of a line work the default delimiter could be accomplished in perl by running:

    perl -ane 'print $F[0], "\n"'
Where $F[0] is the equivalent of $1 in awk.

Re: Awk: The Power and Promise of a 40-Year-Old Language

#113

Post by AstroJetson »

boogies wrote: post #103
nmz wrote: post #82

Earlier quoted context omitted.

CSV is a complicated format but that does not mean awk is incapable of dealing with it. https://www.gnu.org/software/gawk/manual/html_node/Splitting... https://github.com/e36freak/awk-libs/blob/master/csv.awk https://raw.githubusercontent.com/Nomarian/Awk-Batteries/mas...

> CSV is a complicated format Surprisingly and unnecessarily so: > ["DSV"] is to Unix what CSV (comma-separated value) format is under Microsoft Windows and elsewhere outside the Unix world. CSV (fields separated by commas, double quotes used to escape commas, no continuation lines) is rarely found under Unix. > In fact, the Microsoft version of CSV is a textbook example of how not to design a textual file format. It…

This is why I hate CSV files. Trying to reformat huge blocks of data is a job that Awk does well. The associative arrays let you build structures that let you do the heavy lifting. For record processing, Awk should be one of the first tools you look at.

Re: Awk: The Power and Promise of a 40-Year-Old Language

#114

Post by vyuh »

zdwolfe wrote: post #105
vyuh wrote: post #13

"A good programmer uses the most powerful tool to do a job. A great programmer uses the least powerful tool that does the job." I believe this, and I always try to find the combination of simple and lightweight tools which does the job at hand correctly. Awk sometimes proves surprisingly powerful. Just look at the concision of this awk one liner doing a fairly complex job: zcat large.log.gz | awk '{print $0 | "gzip -…

I really love that quote "..A good programmer...", do you have a source?

I believe the source is https://mathwithbaddrawings.com/2015/12/16/good-mathematicia...

Re: Awk: The Power and Promise of a 40-Year-Old Language

#115

Post by fomine3 »

r-bar wrote: post #94
abecedarius wrote: post #80

Earlier quoted context omitted.

A nice thing about awk vs. Perl/Python: there's a small focused set of things to learn. Once you learn them you're done. This suggests an opening for a Perl/Python intro focused on the exact same tasks, admittedly. That seems more realistic for Perl -- unless there's someone who writes Python one-liners at the shell?

I don't think true python "one liners" are a thing, but the awkward thing about awk is sits in this place where what you are doing is complicated enough you need awk, but simple enough you need a one liner? Those cases have been exceedingly few and far between for me enough that every time I want to reach for awk I have to go lookup how to do anything more complex than printing fields. That completely defeats the poi…

Perl is sometimes "better awk".

Re: Awk: The Power and Promise of a 40-Year-Old Language

#116

Post by fomine3 »

zeteo wrote: post #36
melling wrote: post #5

i no longer use it but Perl was always the better solution when one thought AWK was the answer. Perl will do those things where AWK really shines and if the problem got bigger, Perl was easier to deal with.

Perl was built initially as a sed/awk killer but got distracted into trying to take over the world. The interpreter for a language with 100x the number of features will always be slower. Also there's a very clear boundary for when I should use awk by itself, as part of a pipeline, or switch to a better tool. I feel like Perl has the potential to suck me imperceptibly into a huge mess where I spend 80% of my time refa…

Did you have ever found that a Perl oneliner is slow so rewriting by awk meaningful?

Re: Awk: The Power and Promise of a 40-Year-Old Language

#117

Post by kamaal »

Anon84 wrote: post #34
dekhn wrote: post #20

I've used Python almost my entire career, but started with out the UNIX tools. I never found awk interesting, then took a peek at it recently and understood: this was the pre-perl! it had scripting-language hash tables!

PERL was originally advertised as a replacement for “awk and sed”

First version of Perl was a replacement for C+awk+sed.

These are days when things like GC, hashmaps, file operations etc were hard things on Unix.

Re: Awk: The Power and Promise of a 40-Year-Old Language

#118

Post by comex »

pcwalton wrote: post #90
greggyb wrote: post #67

Earlier quoted context omitted.

The POSIX specification includes awk, but not perl or python. The world of UNIX and UNIX-likes is larger than just Linux distributions. Depending on the utility you plan on building and the platforms you expect it to run on, it may be wiser to reach for awk than other PLs.

Modern BSDs, macOS, and Solaris certainly have Perl and Python. (iOS and Android don't, but they don't have awk either.) What other Unixes are you thinking of? AIX, HP/UX, IRIX, UnixWare, etc. should be considered retrocomputing at this point and not relevant to modern compatibility discussions. Linux distros based on busybox, as mentioned elsewhere in this thread, are a more compelling reason for considering awk tha…

When it comes to Python on macOS, the only version that’s installed by default is the deprecated copy of Python 2.7 that’s slated to be removed in the future. For Python 3, you need to install the developer tools. (/usr/bin/python3 ships with the OS, but is just a stub that runs the developer tools version if installed or prompts you to install it otherwise).

It’s not hard to install, but it’s not guaranteed to already be installed on every system.

Re: Awk: The Power and Promise of a 40-Year-Old Language

#119

Post by Akronymus »

boogies wrote: post #103
nmz wrote: post #82

Earlier quoted context omitted.

CSV is a complicated format but that does not mean awk is incapable of dealing with it. https://www.gnu.org/software/gawk/manual/html_node/Splitting... https://github.com/e36freak/awk-libs/blob/master/csv.awk https://raw.githubusercontent.com/Nomarian/Awk-Batteries/mas...

> CSV is a complicated format Surprisingly and unnecessarily so: > ["DSV"] is to Unix what CSV (comma-separated value) format is under Microsoft Windows and elsewhere outside the Unix world. CSV (fields separated by commas, double quotes used to escape commas, no continuation lines) is rarely found under Unix. > In fact, the Microsoft version of CSV is a textbook example of how not to design a textual file format. It…

> The latter conveniently not only handles the separator character, but gives us a way to handle the escape character and newlines for free. CSV, on the other hand, encloses the entire field in double quotes if it contains the separator. If the field contains double quotes, it must also be enclosed in double quotes, and the individual double quotes in the field must themselves be repeated twice to indicate that they don't end the field.

I KNOW how CSV works, for the most part. And my brain still started tuning out/stopped building up the mental model.

Re: Awk: The Power and Promise of a 40-Year-Old Language

#120

Post by nerdponx »

boogies wrote: post #103
nmz wrote: post #82

Earlier quoted context omitted.

CSV is a complicated format but that does not mean awk is incapable of dealing with it. https://www.gnu.org/software/gawk/manual/html_node/Splitting... https://github.com/e36freak/awk-libs/blob/master/csv.awk https://raw.githubusercontent.com/Nomarian/Awk-Batteries/mas...

> CSV is a complicated format Surprisingly and unnecessarily so: > ["DSV"] is to Unix what CSV (comma-separated value) format is under Microsoft Windows and elsewhere outside the Unix world. CSV (fields separated by commas, double quotes used to escape commas, no continuation lines) is rarely found under Unix. > In fact, the Microsoft version of CSV is a textbook example of how not to design a textual file format. It…

The quoting also helps preserve embedded non-printable characters, newlines, etc. (yes, which can appear).

One extension of the "Unix version" would be to impose a requirement like that in JSON, where all non-printable and/or non-ASCII characters must be written as an escape sequence like "\uXXXX" escape.

Post reply on HN