Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Learn Awk? (2016)

blog.jpalardy.com

Post reply on HN
Print view View original

201–210 of 246 posts

Re: Why Learn Awk? (2016)

#201

Post by cyphar »

nrclark wrote: post #135
0xff00ffee wrote: post #132

Earlier quoted context omitted.

Thanks. I forgot GPL is the touch of death in many cases due to how it infects entire codebases. I can't edit my OP but I'm already downvoted so that will suffice.

Specifically GPLv3 is the sticking point - not the GPL in general. GPLv2 is a great license, and I use it for a lot of tools that I write. That's the license that the Linux kernel uses. GPLv3 (which was written in 2007) has much tougher restrictions. It's the license for most of the GNU packages now, and GPLv3 packages are impractical to include in any firmware that also comes with secret sauce. So most of us in the…

That's not an entirely accurate understanding of the GPLv3 "anti-tivoisation" restrictions. The restrictions boil down to "if you distribute hardware that has GPLv3 code on it, you must provide the source code (as with GPLv2) and a way for users to replace the GPLv3 code -- but only if it is possible for the vendor to do such a replacement". There's no requirement to relicense other code to GPLv3 -- if there were then GPLv3 wouldn't be an OSI-approved license.

It should be noted that GPLv2 actually had a (much weaker) version of the same restriction:

> For an executable work, complete source code means all the source code for all modules it contains, plus any associated interface definition files, plus the scripts used to control compilation and installation of the executable. [emphasis added]

(Scripts in this context doesn't mean "shell scripts" necessarily, but more like instructions -- like the script of a play.)

So it's not really a surprise that (when the above clause was found to not solve the problem of unmodifiable GPL code) the restrictions were expanded. The GPLv3 also has a bunch of other improvements (such as better ways of resolving accidental infringement, and a patents clause to make it compatible with Apache-2.0).

Re: Why Learn Awk? (2016)

#203

Post by jabl »

kmstout wrote: post #157
Communitivity wrote: post #109

Awk is a command I turn to time and again. For me it's the single most valuable command for enabling the piped single-purpose pattern. As an example, if I want a sorted list of all open files under the home directories on CentOs I can do this: lsof | awk '{ print $10 }' | grep ^/home/ | sort | uniq

Don' stop there. You've solved a real problem in your life, and you might want that information another day. Make a tiny script that encapsulates it. Generalize it a tiny bit, and give it a memorable name (perhaps lsof-tree). That done, you can stop worrying about the mechanics of the solution and build on it. #! /bin/bash # lsof-tree: list open files in a given directory tree (default /home) NAME=9 # set to 10 for C…

YMMV, but I find it easier and faster to know the basic utilities and how to compose them with pipes than remembering the name of a zillion such scripts.

I guess it depends on how often you need that particular pipeline. Every day? Sure, make a script. Every few months? Nah, I won't remember it anyway,or probably I remember that I've made a script like that but then I have to start searching my bin directory in the end using more time than just writing the pipeline in the first place.

Re: Why Learn Awk? (2016)

#204

Post by rz2k »

graphpapa wrote: post #181
nextos wrote: post #146

Earlier quoted context omitted.

I disagree, it's quite elegant if you think in terms of relational algebra operators: * Projection (Π): awk and cut for simple cases * Selection (σ): grep for simple cases, otherwise sed & awk * Rename (ρ): sed * Set operators: join, comm...

I’m pretty mathsy but I don’t get this

It is from relational algebra used in database theory. There is an excerpt from one of the first MOOCS offered here on Lagunitas now.[1] It is pretty intuitive once you get the hang of it.

[1] https://lagunita.stanford.edu/courses/DB/RA/SelfPaced/course...

Re: Why Learn Awk? (2016)

#205

Post by gbacon »

0xff00ffee wrote: post #87
pizza234 wrote: post #57

Earlier quoted context omitted.

> I don't know of any easier equivalent of "awk '{ print $2 }'" for what it does. I'm not sure if you refer spefically to cut, but Perl has something similar and approximaly terse: > echo 'a b c' | perl -lane 'print $F[1]' Also, Perl can slice arrays, which is something that I really miss in Awk.

PERL is bloatware by comparison and less likely to be installed on distros than AWK. (e.g, embedded or slim distros. that's why you rarely see nonstandard /bin execs in shell scripts).

Do you have data on the relative sizes of the Perl and awk install bases?

Perl is the language, and perl is the implementation. Spelling it with ALL CAPS announces that someone knows little about the language.

Re: Why Learn Awk? (2016)

#206

Post by gbacon »

7thaccount wrote: post #155
0xff00ffee wrote: post #87

Earlier quoted context omitted.

PERL is bloatware by comparison and less likely to be installed on distros than AWK. (e.g, embedded or slim distros. that's why you rarely see nonstandard /bin execs in shell scripts).

Perl used to be part of most distros, but I think favor shifted to Python a few years ago. I wouldn't call it bloat, but yes it is much bigger. At the time you had C (really fast, but cumbersome) and Awk/Bash (good prototyping tools, but not good for large codebases). Perl was the perfect answer to something that is fairly fast, relatively easy to develop in, and easier to write full-sized codebases

Larry Wall referred to the old dichotomy of the “manipulexity” of C for reaching into low-level system details versus the “whipuptitude” of shell, awk, sed, and friends. He targeted Perl for the sweet spot in the unfilled gap between them.

Re: Why Learn Awk? (2016)

#207

Post by kragen »

pizza234 wrote: post #57
jerf wrote: post #9

Because for some bizarre reason, "cut" doesn't ship with any decent column selection logic that is the equivalent of awk's $1, $2, etc., even in 2020. That's like 90% of my use of awk right there. I don't know of any easier equivalent of "awk '{ print $2 }'" for what it does. Posted partially so the Internet Correction Squad can froth at the mouth and set me straight, because I'd love to be showed to be wrong here.

> I don't know of any easier equivalent of "awk '{ print $2 }'" for what it does. I'm not sure if you refer spefically to cut, but Perl has something similar and approximaly terse: > echo 'a b c' | perl -lane 'print $F[1]' Also, Perl can slice arrays, which is something that I really miss in Awk.

If you're golfing, there's also

    echo a b c | perl -pale '$_=$F[1]'

Re: Why Learn Awk? (2016)

#208

Post by gbacon »

shaklee3 wrote: post #188
0xff00ffee wrote: post #87

Earlier quoted context omitted.

PERL is bloatware by comparison and less likely to be installed on distros than AWK. (e.g, embedded or slim distros. that's why you rarely see nonstandard /bin execs in shell scripts).

Unfortunately, for large files perl is significantly faster than awk. I was working on some very large files doing some splitting, and perl was over an order of magnitude faster.

A tool that is stable, well supported, has outstanding documentation, thoroughly tested, won’t capriciously break your code, and outperforms the rest of the pack is not the unfortunate case.

Re: Why Learn Awk? (2016)

#209

Post by gbacon »

Eikon wrote: post #85
umvi wrote: post #23

Earlier quoted context omitted.

I'm not actually convinced unit testing is all that valuable, unless the unit under test has a very clear input -> output transformation (like algorithms, string utilities, etc). If it doesn't (and most units don't), unit tests just encumber you.

I’m really glad to read that. I never understood the whole religion around unit tests. Integration tests are often far easier to write and far more valuable. Like you said, unit tests are really nice when testing for a known, expected output. Unit tests that are effectively testing mocks and crazy stubs because your method has side effects? Not for me.

Integration tests can be easier to write, are necessary, but can also be much slower to run. Yes, it’s possible to write bad unit tests; the same is true of integration tests.

You need both unit tests and integration tests.

Re: Why Learn Awk? (2016)

#210

Post by kragen »

wahern wrote: post #160
eska wrote: post #105

Earlier quoted context omitted.

First of all I'm not a distro maintainer. I also doubt that people would use awk for seriously space constrained environments. And distros ship both awk and python anyway. And again, I don't understand why they'd support networking but not basic data types/functions. The only reason I could've seen to use awk was to throw code together more quickly in a DSL. However this is much less the case than I had hoped. For th…

> I also doubt that people would use awk for seriously space constrained environments. And distros ship both awk and python anyway. Python is absolutely not available everywhere one can find Awk. I've never seen a system with Python but not Awk, but have seen many systems with Awk but not Python (excluding the BSDs, where Python is never in base, anyhow). Actually, not many years ago I used to claim that I never saw…

There was a long period of time where it was easy to find a non-Linux Unix with Perl installed but not Bash: SunOS, Solaris, IRIX, etc., admins would typically install Perl pretty early on, while Bash was more niche. Like, maybe 1990 to 2000. Now we're getting into an era where lots of Unix boxes run MacOS, and although they have Bash, it's a version of only archaeological interest. But they do have Perl.
Post reply on HN