Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Learn Awk? (2016)

blog.jpalardy.com

Post reply on HN
Print view View original

131–140 of 246 posts

Re: Why Learn Awk? (2016)

#131

Post by danwills »

patrickdavey wrote: post #103
dkarl wrote: post #82

Earlier quoted context omitted.

I define aliases c1, c2, c3, c4, etc. in my .bashrc as "awk '{print $1}'" etc. But it's nice to have awk for the slightly more complicated cases, up until it's easier to use Python or another language.

I don't suppose your dotfiles are available anywhere. I'm just wondering what other useful things I can steal ;)

I don't have my dotfile here (on my phone) but here's some ideas from things I've aliased that I use a lot:

cdd : like cd but can take a file path

- : cd - (like 1-level undo for cd)

.. : cd ..

... : cd ../.. (and so on up to '.......')

nd : like cd but enter the dir too

udd : go up (cd ..) and then rmdir

xd : search for arg in parent dirs, then go as deep as possible on the other branch (like super-lazy cd, actually calls a python script).

ai : edit alias file and then source it

Also I set a bindkey (F5 I think) that types out | awk '{print}' followed by the left key twice so that the cursor is in the correct spot to start awk'in ;D

# Bind F5 to type out awk boilerplate and position the cursor

bindkey -c -s "^[[[E" " | awk '{print }'^[[D^[[D"

Edit: better formatting (and at work now so pasted the bindkey)

Re: Why Learn Awk? (2016)

#132

Post by 0xff00ffee »

nrclark wrote: post #104
0xff00ffee wrote: post #101

Earlier quoted context omitted.

Proof? Or are you just guessing?

Can confirm. GNU awk is GPLv3, which means it can't be legally included on any system that prevents a user from modifying the installed firmware. This is a result of GPLv3's "Installation Instructions" requirement. Every commercial embedded Linux product that I've seen uses Busybox (or maybe Toybox) to provide the coreutils. If awk is available on a system like that, it's almost certainly Busybox awk. And Busybox awk…

Thanks. I forgot GPL is the touch of death in many cases due to how it infects entire codebases.

I can't edit my OP but I'm already downvoted so that will suffice.

Re: Why Learn Awk? (2016)

#133

Post by ColinWright »

Scarbutt wrote: post #62
ColinWright wrote: post #37

Earlier quoted context omitted.

I'm an expert on neither AWK nor cut , but AWK allows you to select a field, or the entire line, and then substrings within those. Select characters 3-6 (inclusive) in the second field: $ echo Testing LengthyString | awk '{print substr($2,3,4)}' > ngth If you want to select columns from the entire line, then: $ echo Testing LengthyString | awk '{print substr($0,3,4)}' > stin Is that what you meant?

What cauthon said, cut lets you select range of fields with a simple option, awk doesn't, ex(csv line): cut -d, -f10-30 (selects from field 10 to 30) Not saying this can't be written in awk with more code, but we were talking about field selection ergonomics.

OK, understood. And yes, in my experience AWK is less good at that, cut would definitely be the right tool.

It doesn't detract from the point at hand - which is perfectly valid - but it's worth noting that there's a confusion here with regards the terminology: "fields" vs "columns". I thought they were referring to "columns of characters" whereas the added explanations[0] are about "columns of fields". That makes a difference.

But as I say, yes, I agree that to select a range of columns of fields, especially several fields from each line, is definitely better with cut.

[0] https://news.ycombinator.com/item?id=22109551

Re: Why Learn Awk? (2016)

#134

Post by gnulinux »

peterwwillis wrote: post #122
sremani wrote: post #2

Honorary mention of Taco Bell Programming. (fits this genre). http://widgetsandshit.com/teddziuba/2010/10/taco-bell-progra... Someone ought to write - Zen and the art of Unix tools usage.

(warning, mandatory HN contrarian comment) "This is the opposite of a trend of nonsense called DevOps, where system administrators start writing unit tests and other things to help the developers warm up to them - Taco Bell Programming is about developers knowing enough about Ops (and Unix in general) so that they don't overthink things, and arrive at simple, scalable solutions" It's not possible for developers to kn…

I don't understand this devs don't understand ops nonsense.

> so you waste tons of time and disk space re-getting the same pages, re-looking up the same hostnames, etc. (And that's Ops knowledge...)

If it's my job to write a web scraper, it's absolutely my job to think about/solve this problem.

Is this a new trend thing?

Re: Why Learn Awk? (2016)

#135

Post by nrclark »

0xff00ffee wrote: post #132
nrclark wrote: post #104

Earlier quoted context omitted.

Can confirm. GNU awk is GPLv3, which means it can't be legally included on any system that prevents a user from modifying the installed firmware. This is a result of GPLv3's "Installation Instructions" requirement. Every commercial embedded Linux product that I've seen uses Busybox (or maybe Toybox) to provide the coreutils. If awk is available on a system like that, it's almost certainly Busybox awk. And Busybox awk…

Thanks. I forgot GPL is the touch of death in many cases due to how it infects entire codebases. I can't edit my OP but I'm already downvoted so that will suffice.

Specifically GPLv3 is the sticking point - not the GPL in general. GPLv2 is a great license, and I use it for a lot of tools that I write. That's the license that the Linux kernel uses.

GPLv3 (which was written in 2007) has much tougher restrictions. It's the license for most of the GNU packages now, and GPLv3 packages are impractical to include in any firmware that also comes with secret sauce. So most of us in the embedded space have ditched the GNU tools in our production firmware (even if they're still used to _compile_ that firmware).

Re: Why Learn Awk? (2016)

#136

Post by doctor_eval »

It would be so great if awk had a csv mode. For whatever reason (Excel), CSV seems to be the default text format for field oriented data.

Maybe I’m dumb but I’ve never come up with a separator regex that is quite right.

Re: Why Learn Awk? (2016)

#137

Post by jimbokun »

bloopernova wrote: post #83

I use awk because there's an almost 100% chance that it's going to be installed on any unix system I can ssh into. I use awk because I like to visually refine my output incrementally. By combining awk with multiple other basic unix commands and pipes, I can get the data that I want out of the data I have. I'm not writing unit tests or perfect code, I'm using rough tools to do a quick one-off job. For instance, "mail…

It's far more elegant and concise than any other scripting language I can think of using to accomplish the same thing.

As the article points out, other languages will have a lot more ceremony around opening and closing the file, breaking the input into fields, initializing variables, etc.

Re: Why Learn Awk? (2016)

#138

Post by hawski »

b215826 wrote: post #59
cauthon wrote: post #45

Earlier quoted context omitted.

I don't think so. I think they're referring to cut's ability to select an arbitrary range of columns, e.g. `cut -f 2-7` to select the 2nd through 7th columns, while awk requires you to explicitly enumerate all desired columns, i.e. `awk '{print $2, $3, $4, $5, $6, $7}'

> "while awk requires you to explicitly enumerate all desired columns" Or you can use loops, e.g., echo $(seq 100) | awk '{ for (i = 2; i

It's more characters than manual way. There should be just a built-in function.

Re: Why Learn Awk? (2016)

#139

Post by mistahenry »

jcims wrote: post #120
bloopernova wrote: post #83

I use awk because there's an almost 100% chance that it's going to be installed on any unix system I can ssh into. I use awk because I like to visually refine my output incrementally. By combining awk with multiple other basic unix commands and pipes, I can get the data that I want out of the data I have. I'm not writing unit tests or perfect code, I'm using rough tools to do a quick one-off job. For instance, "mail…

I know you’re not asking for awk protips but you can prefix the block with a match condition for processing. ... | grep foo | awk ‘{print $6}’ | ... becomes ... | awk ‘/foo/{print $6}’ | ... If you start working this into your awk habits you’ll find delightful little edge cases that you can handle with other expressions before the block (you can, for example, match specific fields).

On the other hand, grep can be far faster for searching alone than awk. I almost always use an initial grep for the string that will most reduce the input to the rest of the pipeline. Later, it feels idiomatic to mix in awk with matches like you suggested

Re: Why Learn Awk? (2016)

#140

Post by just_myles »

jcims wrote: post #120
bloopernova wrote: post #83

I use awk because there's an almost 100% chance that it's going to be installed on any unix system I can ssh into. I use awk because I like to visually refine my output incrementally. By combining awk with multiple other basic unix commands and pipes, I can get the data that I want out of the data I have. I'm not writing unit tests or perfect code, I'm using rough tools to do a quick one-off job. For instance, "mail…

I know you’re not asking for awk protips but you can prefix the block with a match condition for processing. ... | grep foo | awk ‘{print $6}’ | ... becomes ... | awk ‘/foo/{print $6}’ | ... If you start working this into your awk habits you’ll find delightful little edge cases that you can handle with other expressions before the block (you can, for example, match specific fields).

Right. I don't consider that particular exhaustive at all and this has helped me when I wanted to do quick searches.
Post reply on HN