Awk: The Power and Promise of a 40-Year-Old Language • Hacker News Board

Re: Awk: The Power and Promise of a 40-Year-Old Language

#41

Post by jrochkind1 » Tue, Sep 07, 2021, 5:30 PM UTC

My first job getting paid to program was in awk. Processing log files.

In the middle of that job, my supervsior, you know what, we're doing increasingly complicated things with awk and it's getting increasingly hacky... I've heard that Perl is like awk but better, do you want to learn Perl and switch to that?

And so we did. My thought then was there was little that was easier in awk than Perl, you could use Perl very much like awk if you wanted, you can even use the right command-line args to have Perl have an "implied loop" like awk... but then you can do a lot more with Perl too.

I don't use Perl anymore. Or awk.

Re: Awk: The Power and Promise of a 40-Year-Old Language

#42

Post by Folcon » Tue, Sep 07, 2021, 5:31 PM UTC

dunefox wrote: post #37

vyuh wrote: post #13

"A good programmer uses the most powerful tool to do a job. A great programmer uses the least powerful tool that does the job." I believe this, and I always try to find the combination of simple and lightweight tools which does the job at hand correctly. Awk sometimes proves surprisingly powerful. Just look at the concision of this awk one liner doing a fairly complex job: zcat large.log.gz | awk '{print $0 | "gzip -…

Ehh. Until the 'job' gets extended and then your simple tool makes it exponentially more complex and you have to rewrite it with the more powerful tool.

Surely that isn't a weakness of a simple tool?

A 5 min job that probably won't get extended saving you from having to spend 20 mins coding something up is better than, feeling annoyed that you have spent the 20 mins coding up the original implementation and then extend it.

Hopefully, you also get the benefit of additional knowledge on that future implementation as well. Why wouldn't this just be a net win?

Unless you're talking about writing hack after hack after hack, eventually leaving yourself with some incomprehensible eldritch monstrosity, in which case, don't do that?

Re: Awk: The Power and Promise of a 40-Year-Old Language

#43

Post by rakoo » Tue, Sep 07, 2021, 5:32 PM UTC

vyuh wrote: post #13

"A good programmer uses the most powerful tool to do a job. A great programmer uses the least powerful tool that does the job." I believe this, and I always try to find the combination of simple and lightweight tools which does the job at hand correctly. Awk sometimes proves surprisingly powerful. Just look at the concision of this awk one liner doing a fairly complex job: zcat large.log.gz | awk '{print $0 | "gzip -…

If I understand this correctly, it will gzip every line separately instead of gzipping them together... it's not really the most effective but it does work

Re: Awk: The Power and Promise of a 40-Year-Old Language

#44

Post by linuxlizard » Tue, Sep 07, 2021, 5:41 PM UTC

I use awk to auto-generate C header files from other header files. I work with $vendor's huge complicated kernel driver codebase. I need small pieces of $vendor's interconnected header files in order to make kernel calls to their drivers without pulling in all their code.

Re: Awk: The Power and Promise of a 40-Year-Old Language

#45

Post by linuxlizard » Tue, Sep 07, 2021, 5:43 PM UTC

jrochkind1 wrote: post #41

My first job getting paid to program was in awk. Processing log files. In the middle of that job, my supervsior, you know what, we're doing increasingly complicated things with awk and it's getting increasingly hacky... I've heard that Perl is like awk but better, do you want to learn Perl and switch to that? And so we did. My thought then was there was little that was easier in awk than Perl, you could use Perl very…

I think I remember reading somewhere Larry Wall was inspired to create Perl in order to combine awk+sed functionality. He was sick of awk+sed being almost powerful enough to do what he needed. (I can't find a reference to this though.)

Re: Awk: The Power and Promise of a 40-Year-Old Language

#46

Post by chasil » Tue, Sep 07, 2021, 5:45 PM UTC

coliveira wrote: post #17

melling wrote: post #5

i no longer use it but Perl was always the better solution when one thought AWK was the answer. Perl will do those things where AWK really shines and if the problem got bigger, Perl was easier to deal with.

The problem is that awk is a very simple language, which you can learn in an afternoon. Perl is a very complex language, and is not used anymore, so you're just spending your time on something you'll rarely use.

OpenBSD's binary package system is written in perl.

Re: Awk: The Power and Promise of a 40-Year-Old Language

#47

Post by jhbadger » Tue, Sep 07, 2021, 5:58 PM UTC

torcete wrote: post #31

I use awk constantly in bioinformatics, for many of the file formats designed to store genomic data, awk is the easiest tool you can use for processing.

There's even a version of awk specifically designed for bioinformatics that natively knows how to handle fasta, fastq, and sam files, among other formats.

https://github.com/lh3/bioawk

Re: Awk: The Power and Promise of a 40-Year-Old Language

#48

Post by WhatIsDukkha » Tue, Sep 07, 2021, 5:59 PM UTC

exdsq wrote: post #23

justin_oaks wrote: post #7

I only recently learned Awk enough to be useful. But I still don't reach for it when I probably should. What are the most common cases where you reach for Awk instead of some other tools? I recently used it to parse and recombine data from the OpenVPN status file. That file has a few differently formatted tables in the same file. Using Awk, I was able to change a variable as each table was encountered, this I could c…

I had to take large CSV files like {question, right_ans, wrong_ans1, wrong_ans2, wrong_ans3} and covert them into SQL insert files. Few caveats - some could be duplicates, some characters were not allowed, and some had formatting issues. The first issue was avoided by upserting, but the other two I used Awk and Sed for and put together a fairly robust script far quicker than if I reached for Python. I probably would…

"""I probably would have reached for Python if I realised how many edge cases there were"""

This is the counter for all the "success" stories of awk users that walked away with an underspecced and underdeveloped 5 minute solution.

Re: Awk: The Power and Promise of a 40-Year-Old Language

#49

Post by cogman10 » Tue, Sep 07, 2021, 6:04 PM UTC

phkahler wrote: post #25

I never use Awk until last year. I wanted to monitor an embedded device with little more than bustbox and python on it. There was quite a bit of information in the log files (I had already written a custom log file viewer with some highlighting) but I wanted to monitor in real-time. Somehow I decided to use Awk to monitor the tail of the log file and do realtime bar-graphs by generating appropriate cursor control seq…

The biggest reason to learn AWK, IMO, is that it's on pretty much every single linux distribution.

You might not have perl or python. You WILL have AWK. Only the most minimal of minimal linux systems will exclude it. Even busybox includes awk. That's how essential it's viewed.

Re: Awk: The Power and Promise of a 40-Year-Old Language

#50

Post by throwawayboise » Tue, Sep 07, 2021, 6:13 PM UTC

chasil wrote: post #46

coliveira wrote: post #17

Earlier quoted context omitted.

The problem is that awk is a very simple language, which you can learn in an afternoon. Perl is a very complex language, and is not used anymore, so you're just spending your time on something you'll rarely use.

OpenBSD's binary package system is written in perl.

Probably as much for legacy reasons as anything else. Perl was the chosen scripting language for utilities, it works, they understand it, and they've kept with it. Sort of how they stay with CVS for their source repository.

Python isn't even installed on a base OpenBSD system.