Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Learn Awk? (2016)

blog.jpalardy.com

Post reply on HN
Print view View original

231–240 of 246 posts

Re: Why Learn Awk? (2016)

#231

Post by thomas2718 »

nextos wrote: post #146
bloopernova wrote: post #83

I use awk because there's an almost 100% chance that it's going to be installed on any unix system I can ssh into. I use awk because I like to visually refine my output incrementally. By combining awk with multiple other basic unix commands and pipes, I can get the data that I want out of the data I have. I'm not writing unit tests or perfect code, I'm using rough tools to do a quick one-off job. For instance, "mail…

I disagree, it's quite elegant if you think in terms of relational algebra operators: * Projection (Π): awk and cut for simple cases * Selection (σ): grep for simple cases, otherwise sed & awk * Rename (ρ): sed * Set operators: join, comm...

Indeed, and with a bit of tuning (e.g., using mawk for most things), one can get quite good performance. [1] The project also provides a translator from Datalog to bash scripts [2].

Disclaimer: I was one of the authors

[1] https://www.thomasrebele.org/publications/2018_report_bashlo... [2] https://www.thomasrebele.org/projects/bashlog/datalog

Re: Why Learn Awk? (2016)

#232

Post by 3xblah »

Annatar wrote: post #221
3xblah wrote: post #198

Earlier quoted context omitted.

If I am not mistaken, -v is GAWK only.

Every contemporary AWK supports -v. Real AWK from UNIX®️ supported -v since at least the '80's.

True. But there are differences when -v is used, as opposed to FS. Try this, where "nawk" is Lucent awk used by BSD

     cat > 1.awk

Re: Why Learn Awk? (2016)

#233

Post by downerending »

0xbadcafebee wrote: post #115
jerf wrote: post #9

Because for some bizarre reason, "cut" doesn't ship with any decent column selection logic that is the equivalent of awk's $1, $2, etc., even in 2020. That's like 90% of my use of awk right there. I don't know of any easier equivalent of "awk '{ print $2 }'" for what it does. Posted partially so the Internet Correction Squad can froth at the mouth and set me straight, because I'd love to be showed to be wrong here.

The Internet Correction Squad would like to remind you that 1) they are different programs that do different things, 2) if they changed over time, they wouldn't be portable, 3) if all you use awk for is '{print $2}', that is perfectly fine. You can submit a new feature request/patch to GNU coreutils' cut , but they'll probably just tell you to use awk. Edit: Nevermind, it's already a rejected feature request: https:/…

One of the bad things about having an ultra-stable core of GNU utils as that they've largely ossified over time. Even truly useful improvements can often no longer get in.

It's a sharp and not-entirely-welcome change from the 80s and 90s.

Here's another that would be great but will never be added: I want bash's "wait" to take an integer argument, causing it to wait until only that number (at most) of background processes are still running. That would make it almost trivial to write small shell scripts that could easily utilize my available CPU cores.

Re: Why Learn Awk? (2016)

#234

Post by just_myles »

shpeedy wrote: post #119
just_myles wrote: post #58

Earlier quoted context omitted.

Because syntatically as a language/tool it is super easy to remember. Writing one liners with awk feels more intuitive to me. Awk example: ls -l | awk '{print $9, $5}' or ls -lh | awk '{print $9, $5}' Seems a whole lot simpler. To me. I find if you have to write exhaustive shell scripts then maybe you can look for something more verbose like Perl, I guess.

Yep, but you have bug in your awk one-liner.

In my defense I did this fairly quickly (Which was the point.) and was not trying to illustrate proper syntax (I mean it does run and does produces an output.).

ls -l | awk '{print $9 "\t" $5}'

That is about as much as i'm willing to do for this.

Re: Why Learn Awk? (2016)

#235

Post by just_myles »

demiol wrote: post #218
wahern wrote: post #163

Earlier quoted context omitted.

If you mean the lack of quotations, then the behavior is well-defined and is presumably what was intended. Per POSIX, > The print statement shall write the value of each expression argument onto the indicated output stream separated by the current output field separator (see variable OFS above), and terminated by the output record separator (see variable ORS above). The default value for OFS is and for ORS, .

> If you mean the lack of quotations, No, lack of commas in output and broken filenames with spaces.

I see your point regarding spaces in names. Suppose I could use FILENAME. But I think my point was made.

Re: Why Learn Awk? (2016)

#236

Post by Communitivity »

kmstout wrote: post #157
Communitivity wrote: post #109

Awk is a command I turn to time and again. For me it's the single most valuable command for enabling the piped single-purpose pattern. As an example, if I want a sorted list of all open files under the home directories on CentOs I can do this: lsof | awk '{ print $10 }' | grep ^/home/ | sort | uniq

Don' stop there. You've solved a real problem in your life, and you might want that information another day. Make a tiny script that encapsulates it. Generalize it a tiny bit, and give it a memorable name (perhaps lsof-tree). That done, you can stop worrying about the mechanics of the solution and build on it. #! /bin/bash # lsof-tree: list open files in a given directory tree (default /home) NAME=9 # set to 10 for C…

Spot on. My ~/bin has a large number of such little scripts, and my current project has a 'scripts' repository for the same reason.

Re: Why Learn Awk? (2016)

#237

Post by nrclark »

cyphar wrote: post #201
nrclark wrote: post #135

Earlier quoted context omitted.

Specifically GPLv3 is the sticking point - not the GPL in general. GPLv2 is a great license, and I use it for a lot of tools that I write. That's the license that the Linux kernel uses. GPLv3 (which was written in 2007) has much tougher restrictions. It's the license for most of the GNU packages now, and GPLv3 packages are impractical to include in any firmware that also comes with secret sauce. So most of us in the…

That's not an entirely accurate understanding of the GPLv3 "anti-tivoisation" restrictions. The restrictions boil down to "if you distribute hardware that has GPLv3 code on it, you must provide the source code (as with GPLv2) and a way for users to replace the GPLv3 code -- but only if it is possible for the vendor to do such a replacement ". There's no requirement to relicense other code to GPLv3 -- if there were th…

I do appreciate the intention behind GPLv3. And it does has a lot of good updates over GPLv2.

The reason why I said it's impractical to include GPLv3 code in a system that also has secret sauce (maybe a special control loop or some audio plugins) is more about sauce protection.

If somebody has access to replace GPLv3 components with their own versions, then they effectively have the ability to read from the filesystem, and to control what's in it (at least partially).

So if I had parts that I wanted to keep secret and/or unmodifiable (maybe downloaded items from an app store), I'd have to find some way to separate out anything that's GPLv3 (and also probably constrain the GPLv3 binaries with cgroups to protect against the binaries being replaced with something nefarious). Or I'd have to avoid GPLv3 code in my product. Not because it requires me to release non-GPL code, but more because it requires me to provide write access to the filesystem.

And I guess that maybe GPLv3 is working as intended there. Not my place to judge if the license restrictions are good or bad. But it does mean that GPLv3 code can't easily be shipped on products that also have files which the developer wants to keep as a trade secret (or files that are pirateable). With the end result that most GNU packages become off-limits to a lot of embedded systems developers.

Re: Why Learn Awk? (2016)

#238

Post by Annatar »

3xblah wrote: post #232
Annatar wrote: post #221

Earlier quoted context omitted.

Every contemporary AWK supports -v. Real AWK from UNIX®️ supported -v since at least the '80's.

True. But there are differences when -v is used, as opposed to FS. Try this, where "nawk" is Lucent awk used by BSD cat > 1.awk

That is not how FS is set; It's set with -F. And there is actually no need to use -v, passing variables at the end works consistently across all AWK's and always has:

  echo "" | awk '{print Bla;}' Bla="Hello."

Re: Why Learn Awk? (2016)

#239

Post by 3xblah »

Annatar wrote: post #238
3xblah wrote: post #232

Earlier quoted context omitted.

True. But there are differences when -v is used, as opposed to FS. Try this, where "nawk" is Lucent awk used by BSD cat > 1.awk

That is not how FS is set; It's set with -F. And there is actually no need to use -v, passing variables at the end works consistently across all AWK's and always has: echo "" | awk '{print Bla;}' Bla="Hello."

What if you set FS with -F but then later in the script want to change FS to something else.

Re: Why Learn Awk? (2016)

#240

Post by canada_dry »

asicsp wrote: post #197
LinuxBender wrote: post #7

Here are some clever one-liners for awk [1] Please be sure to add your own. [1] - https://www.commandlinefu.com/commands/matching/awk/YXdr/sor...

I have a repo dedicated for some of the cli text processing tools like grep/sed/awk/perl/sort/etc. Here's my one-liner collection for awk [1] [1] https://github.com/learnbyexample/Command-line-text-processi...

Wow... a great resource! It's very well written and consistent.
Post reply on HN