91–100 of 136 posts
Perhaps my old sysadmin hat is showing through, but I don’t quite see what the advantage of awk is over just writing the same thing in perl. I’ve seen my fair share of horrendous shell scripts from junior sysadmins, and every time I think to myself “the text processing portion would be so much cleaner in Perl”.
Awesome! I've been meaning to replace my usage of Python/JavaScript for tasks (which I believe) are more awk-shaped.
It’s usually the opposite direction that you mentioned that you want to go. You one liner some shell like awk to quickly get shit done without worrying about a runtime being available to you and then if you need it to be more robust and legible because of testing etc or production grade you move to a proper dynamic scripting environment
I love awk, and I find myself reaching for it a fair bit. One of the main things I use it for is “sed with state,” so for things like matching on a line, but only if it was preceded by some other line. I find this to be really useful for creating one-off linters, for example I made one recently to check all our migration files for CREATE INDEX without CONCURRENTLY on a particular set of very large tables where it wou…
sed -ne 'x' -e '/PREV/ {x; /CURR/ p; x}'
> echo -e "PREV\nCURR\nCURR\nCURR\nPREV\nRED" | sed -ne 'x' -e '/PREV/ {x; /CURR/ p; x}'
CURR
sed -n
sed -ne 'x'
sed -ne 'x' -e '/PREV/
sed -ne 'x' -e '/PREV/ { ... }'
sed -ne 'x' -e '/PREV/ { x; ... }'
sed -ne 'x' -e '/PREV/ { x; /CURR/ p; ... }'
sed -ne 'x' -e '/PREV/ {x; /CURR/ p; x}'
All that said, I'm pretty sure the `awk` script is much simpler and more direct, but I wanted to share how one might accomplish this was sed.
The time I spent learning this probably would've been better spend on awk, but this tutorial[0], was so good and so easy, it taught me nearly everything I know about sed.
Earlier quoted context omitted.
If you are comparing Awk vs Perl for scripts, I'd prefer Perl (or Python). This post is about short one-liners for ad hoc use cases. I prefer sed/awk over Perl for such cases. Though, if you already know Perl, you could continue using it instead of having to learn more tools.
Do all systems still come with Perl baked in these days? If so I could see reaching for that over awk/sed. If I have to install a runtime I may as well just reach for Python
Perhaps my old sysadmin hat is showing through, but I don’t quite see what the advantage of awk is over just writing the same thing in perl. I’ve seen my fair share of horrendous shell scripts from junior sysadmins, and every time I think to myself “the text processing portion would be so much cleaner in Perl”.
As a LANGUAGE, it's "eh". It just happens to be "good enough".
You can, of course, do all of that with Perl. But then I have to write all that boiler plate I get with awk for free. And the gains in Perls language aren't enough, for me, to dump awk. And I don't use it for "scripting", I use it for data processing, tearing up files for mostly one off tasks. So I don't miss Perls depth. If I want depth, I'll go somewhere else.
What is better? Starting with awk or sed?
Perhaps my old sysadmin hat is showing through, but I don’t quite see what the advantage of awk is over just writing the same thing in perl. I’ve seen my fair share of horrendous shell scripts from junior sysadmins, and every time I think to myself “the text processing portion would be so much cleaner in Perl”.
Not being snarky, why not python over perl? what makes perl better for scripts?
Having an implicit line- and field-splitting loop for standard input with a couple of command-line switches. (Awk doesn't even need switches, but is cumbersome if you need initial state.) This covers a lot of use-cases. Also, very compact and powerful regular expressions.
What is better? Starting with awk or sed?
But I learned awk while sitting in an office at a client site. I forget the specific scenario, but I wanted to split up some files into some other files. I didn't even know awk, but grokked enough from the man page to let me do what I wanted to do. I can't even say what provoked me to turn to awk in the first place. I do know I ran into some internal open file limits, but worked around that.
If you want to tear files apart, or summarize them in some way, or push the fields around, awk is much better. sed is an editor. If I have a sed scenario, I'm more apt to just do it in vi and save the result than stitch together some pipeline with sed.
Most of my use cases are one off processing and analysis. I've never had any workflows that relied on awk or most anything like that. It was almost all throw away code, a tool on the workbench, not the production line.
Earlier quoted context omitted.
You can do it with cut too: $ echo "key: value" | cut -wf 2 value but whether it's actually "simpler" is open to debate edit: actually gnu cut lacks -w, so this is bsd-only. lol computers, stick with awk
I can't tell you how many times I pipe in rev to put my text where I want it for cut (then rev it again). Abbreviated example, getting the service names from a k8s cluster looks roughly like (actual command does a bit more processing): kubectl get deployments -o wide | rev | cut -d'=' -f1 | rev But if it's just gobbling whitespace, xargs without a command can be your friend. $ echo "key: value" | cut -d: -f2 | xargs…
For the sake of the argument, say I have the following fixed output and want the sizes:
$ ls -l
-rw-rw-r-- 1 userAAA group 588 Aug 29 00:25 file1
-rw-rw-r-- 1 userAA groupB 11870 Aug 29 00:24 file2
-rw-rw-r-- 1 userA groupBB 1166 Aug 28 23:56 file3
-rw-rw-r-- 1 user groupBBB 195 Aug 28 23:56 file4
$ ls -l | awk '{print $5}'I love awk, and I find myself reaching for it a fair bit. One of the main things I use it for is “sed with state,” so for things like matching on a line, but only if it was preceded by some other line. I find this to be really useful for creating one-off linters, for example I made one recently to check all our migration files for CREATE INDEX without CONCURRENTLY on a particular set of very large tables where it wou…
One of these days I need to get around to learning awk. In the meantime, I've learned some of the deeper, stateful, features of sed. For instance, you mentioned wanting to only output a line if it was preceded by another. Here's a sed command that does so: sed -ne 'x' -e '/PREV/ {x; /CURR/ p; x}' > echo -e "PREV\nCURR\nCURR\nCURR\nPREV\nRED" | sed -ne 'x' -e '/PREV/ {x; /CURR/ p; x}' CURR This uses sed's hold buffer.…
Plan9's awk(1)[0] man page provides a precise and concise (a few paragraphs) presentation of the core features of all awk implementations.
Tutorials bring practical knowledge, but often lack complete and self-contained descriptions of those nifty little tools.