Live data from Hacker News

Skip grep, use awk

blog.jpalardy.com

81–90 of 136 posts

Re: Skip grep, use awk

#82
post #63

It's a nice idea, but I think maybe only to replace a simple egrep. grep has tons of great option like -F = fgrep; no regex - way faster -v = as mentioned in the article -o = print only matched input, not the entire line -C = context, print lines before and after the match; can also be used partially with -A (after) and -B (before)

Small note: For GNU grep on single regexes at least, the -F flag should not impact performance. It is smart enough to see through a pattern as a literal and avoid the regex engine.

Thanks, I didn't know this.

Re: Skip grep, use awk

#83
post #29

How am I supposed to do `grep -nrb --include \*.cpp something` ?

This is close, but I don't know how to implement grep's `-b` easily:

    find . -type f -name "*.cpp" -exec awk '/something/ {print FILENAME, NR, $0}' {} \+
Clearly grep wins this round!

Re: Skip grep, use awk

#84
post #53

Earlier quoted context omitted.

As per other users, you can use pkill but it's not entirely portable in meaning between OSs. This software had to run on just about every possible Unix out there. Solaris, HP-UX, AIX, many others ... My main point is that using perl and grep, (multiple thereof!) is nuts when you can do it all in perl. Also crazy was using -ef (all processes) when the process of interest was using a known user. So ps -u would be more…

If it's something you need to do on a semi-regular basis, using a pidfile seems to me to be the best solution. Instead of trying to be clever about pipelines, just write the pid(s) to a file and use that.

That's the right way to do it but you always need a backup method to use in case the process dies without removing the pidfile.

Re: Skip grep, use awk

#85
post #45

Earlier quoted context omitted.

That's what I was referencing when I said > Pulling out a field from a line is what most people use awk for, but it's honestly the least interesting part of awk. In fact, if cut supported regular expressions for specifying the field and record separators people wouldn't be using awk for that purpose (because that's all that $n does). There's nothing magical about awk's default FS. It's literally just /\s+/. If cut's…

I wish the default 'cut' implementation could be just a little more clever - regex delimiters would be good, it doesn't even support multiple characters :( Also, cut's output manipulation is surprising. '-f 2,1' is actually the same as '-f 1,2' - you can't change the order of printing. I know there are other programs that can do the job, but it's a little frustrating when you can 'almost' get there with a chain of pi…

You mean to say you don't use

    some-cmd | sed -E 's/^\s+//;s/\s+/ /g' | cut -f$n
Rather than

    some-cmd | awk '{ print $n }'
;) Also I agree it's silly you can't change the field order.

Re: Skip grep, use awk

#86
post #22

Earlier quoted context omitted.

awk is a full programming language, and most of the times that I'm doing awk scripting I have to use things like associative arrays and arithmetic. Pulling out a field from a line is what most people use awk for, but it's honestly the least interesting part of awk. In fact, if cut supported regular expressions for specifying the field and record separators people wouldn't be using awk for that purpose (because that's…

Yeah, ripgrep is "a grep," not an awk. I'm not sure how they wound up being conflated here. ripgrep does have a `-r/--replace` flag which is somewhat of a generalization of grep's `-o/--only-matching` flag (and also part of ack, I believe) by permitting sub-capture expansion that probably does replace awk for the "pulling out a field from a line" use case you mentioned. But it's pretty awkward for simple cases. e.g.,…

>Yeah, ripgrep is "a grep," not an awk. I'm not sure how they wound up being conflated here.

Simple: a lot of people use awk just as a grep.

Re: Skip grep, use awk

#88
Skip awk, use perl....

The alias below sets perl to loop over STDIN splitting each line on more than one whitespace character and populate an array F. The -nE will then Evaluate an expression from the command line looping over the input line-by-line.

    alias glorp='perl -aF"/\s+/" -nE'
So now we have the command `glorp` to play with which has more familiar syntax than awk and all of CPAN available to play with!

    $ [data is generated] | glorp '/Something/ and say $F[2]'
We have access to any Perl module by putting -MModule::Name=function after the command, the following will parse a JSON record per line and glorp out what we wanted:

    $ echo -e '{"hello":"world"}\n{"hello":"cat"}' | glorp 'say decode_json($_)->{hello};' -MJSON=decode_json
    world
    cat
Maybe you are used to using curl too. There is a nice web framework in Perl called Mojolicious (http://mojolicious.org) that provides a convenience module called 'ojo' for command line use. So grabbing the summary sentence from Wikipedia articles is as straight forward as below. Notice Mojolicious lets us use CSS selectors!

    $ echo -e 'grep\nawk\nperl' \
      | glorp 'say g("wikipedia.org/wiki/$F[0]")->dom->at("#mw-content-text > div > p")->all_text' -Mojo

Re: Skip grep, use awk

#89
post #74

Earlier quoted context omitted.

ripgrep doesn't yet support compressed files (z/gz), while grep/ack/ag do. On *nix it's extremely common to have sparsely compressed directories, from logfiles to non-changing documentation. I really wished ripgrep would just stream to a fast coprocess and support any stream compressor transparently instead of trying to use a built-in rust library.

Suggestions are most welcome on the issue tracker. I don't think your idea has been suggested yet?

I have commented on #225 about this.

Re: Skip grep, use awk

#90
awk is certainly a really important tool to know, but in general this is poor advice. awk is significantly slower than grep -E, and typing awk commands is often much slower as well. Not to mention that awk only operates on a single stream of data, and can't do operations with file awareness. Sure, use awk's line filtering when you're already going to need awk for something else, but in general your first instinct should be grep.
Post reply on HN