Earlier quoted context omitted.
Thanks. I forgot GPL is the touch of death in many cases due to how it infects entire codebases. I can't edit my OP but I'm already downvoted so that will suffice.
Specifically GPLv3 is the sticking point - not the GPL in general. GPLv2 is a great license, and I use it for a lot of tools that I write. That's the license that the Linux kernel uses. GPLv3 (which was written in 2007) has much tougher restrictions. It's the license for most of the GNU packages now, and GPLv3 packages are impractical to include in any firmware that also comes with secret sauce. So most of us in the…
Why Learn Awk? (2016)
161–170 of 246 posts
Re: Why Learn Awk? (2016)
#162Earlier quoted context omitted.
> I don't know of any easier equivalent of "awk '{ print $2 }'" for what it does. Does `cut -f2` not work? My complaint with cut is that you can't reorder columns (e.g. `cut -f3,2` ) Awk is really great for general text munging, more than just column extraction, highly recommend picking up some of the basics Edit to agree with the commenters below me: If the file isn't delimited with a single character, then cut alon…
It does not. Compare: $ echo ' 1 2 3' | cut -f2 1 2 3 $ echo ' 1 2 3' | cut -f2 -d' ' $ echo ' 1 2 3' | awk '{print $2}' 2 "-f [...] Output fields are separated by a single occurrence of the field delimiter character."
or
echo ' 1 2 3' | tr -s ' ' '\t' | cut -b 2- | cut -f2
Re: Why Learn Awk? (2016)
#163Earlier quoted context omitted.
Because syntatically as a language/tool it is super easy to remember. Writing one liners with awk feels more intuitive to me. Awk example: ls -l | awk '{print $9, $5}' or ls -lh | awk '{print $9, $5}' Seems a whole lot simpler. To me. I find if you have to write exhaustive shell scripts then maybe you can look for something more verbose like Perl, I guess.
Yep, but you have bug in your awk one-liner.
> The print statement shall write the value of each expression argument onto the indicated output stream separated by the current output field separator (see variable OFS above), and terminated by the output record separator (see variable ORS above).
The default value for OFS is and for ORS, .
Re: Why Learn Awk? (2016)
#164Re: Why Learn Awk? (2016)
#165Earlier quoted context omitted.
Perl is much faster[0], with much more features, with bunch of ready to use libraries, with package manager (CPAN), and similar syntax to awk. Why you use awk? [0]: http://rc3.org/2014/08/28/surprisingly-perl-outperforms-sed-...
Perl was my second programming love, but awk is much shorter and easier to remember for the simple cases where I need it. Remembering which Perl command-line arguments simulate awk’s line-by-line processing is harder than just remembering awk.
I think if you know Perl really well and can remember the command line arguments - particularly -E, -n, -I and -p - then it’s a good swap in substitute for grep, sed, awk, cut, bash, etc when whatever 5 min task you’re working on gets a tiny bit more complex.
Similarly a decent version perl 5 seems to be installed everywhere by default.
I’m curious to know if anyone would say the same about python or any other programs? I’m not particularly strong in short python scripting.
Re: Why Learn Awk? (2016)
#166Hmm. I'm sure this question will induce a flamewar of practical "#NeverAwk"-ers fighting toolbelt bloat, versus tech-hoarding AWK apologists arguing against throwing something out given if fills . Here's the thing: these arguments all too commonly focus on subjective notions of "simplicity", and toy examples divorced from actual common practise, and or solid comparable benchmarks . Show me a range of practical exampl…
I think your missing the point of awk. The O'Reilly sed and awk book has some complex examples, but when I look at my own usage they are all toy examples within a much larger scope. It's more like a special DSL extension for my shell than something I'd pick to build the entire solution, so a comparison to perl, python and ruby don't really make sense, they are general purpose languages but awk just has a couple of features that make it a very specialized yet useful one.
As an example, a have a system for importing and parsing log files that mostly done from a shell script, awk is used in two parts. The first is to transform a structured and easy to read file (records '\n\n' separated) into a csv easier to consume for bash, there's probably quite a few options to do this from tr to bash and it's done inline. The second is to filter the results down to what I need, so I have scripts like:
#!/usr/bin/awk -f
/some common error I don't care about/ { next } #skip line
/other common error/ { next }
/Error/ { print $0 } #this prints error lines, alternatively:
/Error/ && !errors[gensub($1", "", "g", $0)]++ { print $0 } #print each error once
{next} #skip everything else
Apart from one single line which wasn't in the original that's something you could teach a total novice in minutes, the /pattern/{action} syntax is about as simple as programming can be. Execution speed could probably be improved with a specific program but I suspect the bottleneck would be the spinning disk anyway, I run this over hundreds of MB every few minutes and it's not a problem, when I run it manually it's near instantaneous, I spend longer waiting for the desktop calculator to open up these days.Re: Why Learn Awk? (2016)
#167Re: Why Learn Awk? (2016)
#168Earlier quoted context omitted.
I know you’re not asking for awk protips but you can prefix the block with a match condition for processing. ... | grep foo | awk ‘{print $6}’ | ... becomes ... | awk ‘/foo/{print $6}’ | ... If you start working this into your awk habits you’ll find delightful little edge cases that you can handle with other expressions before the block (you can, for example, match specific fields).
To pile on :-) you often want -w (match word) flag to grep. In awk, I couldn't find how to do this. I tried /\bfoo\b/ and /\ / but neither worked. I don't know why and don't care enough which brings me to my major awk irritation ... It doesn't use extended or perl REs, which makes it quite different to ruby, perl, python, java. Now, according to the man page it does ; at least on OSX (man re_format) but as mentioned…
Re: Why Learn Awk? (2016)
#169Awk, like shell, is a fraught programming environment, full of silent failure and hidden gotchas. Even for one-liners. I use shell because I have to, not because I like it. I dread maintaining shell scripts which have a bunch of awk and sed in them. The Unix ideal of small single-purpose tools and text processing is separable from these old warhorses.
So for example, selecting columns is easy, and can be done by name.
Here's a useful little snippet demonstrating converting and processing the CSV output of the legacy "whoami" Windows command. It lists the groups a user is a member of without poking domain controllers using LDAP. It always works, including cross-forest scenarios.
WHOAMI /GROUPS /FO CSV `
| ConvertFrom-Csv `
| Where-Object Attributes -NotLike '*used for deny only*' `
| Select-Object -ExpandProperty 'Group Name'
It looks like a lot of typing, but everything tab-completes and the end-result is human readable. I find that people that prefer terseness over verbose syntax are selfish. They simply don't care about the future maintainers of their scripts.PowerShell can also natively parse JSON and XML into object hierarchies and then select from them. That's difficult in UNIX. The Invoke-RestMethod command is particularly brilliant, because instead of a stream of bytes it returns an object that can be manipulated directly.
$p = Invoke-RestMethod 'https://ipapi.co/8.8.8.8/json/'
echo $p.org
echo $p.country_name
PowerShell Core is available on Linux and is great for ad-hoc tabular data processing! Give it a go...Re: Why Learn Awk? (2016)
#170Earlier quoted context omitted.
Proof? Or are you just guessing?
Can confirm. GNU awk is GPLv3, which means it can't be legally included on any system that prevents a user from modifying the installed firmware. This is a result of GPLv3's "Installation Instructions" requirement. Every commercial embedded Linux product that I've seen uses Busybox (or maybe Toybox) to provide the coreutils. If awk is available on a system like that, it's almost certainly Busybox awk. And Busybox awk…
The point of the license condition is that once a device has been sold the new owner should have as much control as the developer to change that specific device. If no one can update it then the condition is fulfilled.