Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Skip grep, use awk

blog.jpalardy.com

Post reply on HN
Print view View original

41–50 of 136 posts

Re: Skip grep, use awk

#41

Post by vidoc »

thaumaturgy wrote: post #19
emmelaich wrote: post #13

My favourite bad example of using grep was from a big enterprise software vendor to kill one of their processes. It looked something like ps -ef| grep SomeDaemon | grep -v grep | grep -v perl | perl -e ' '

Serious question: what would be a better way to do this?

I often roll with 'pgrep' part of pkill package to fetch the pids of either the ucomm or longname (-l) of the process.

pgrep -l 'reg.*ex" | xargs -L1 do-stuff

or just pkill if the idea is just to send a signal.

To gather particular process metrics, ps can be invoked with process ids (-p), with full control of its output (-o), grep is rarely needed.

Re: Skip grep, use awk

#42

Post by jsmthrowaway »

CaptSpify wrote: post #31
thaumaturgy wrote: post #19

Earlier quoted context omitted.

Serious question: what would be a better way to do this?

I would just recommend pkill: https://en.wikipedia.org/wiki/Pkill

pgrep is also useful if you're not trying to kill. They're both from Solaris, and if you have one, you have the other. pgrep | xargs can also be very useful on occasion, depending on what you're doing (mainly debugging).

Re: Skip grep, use awk

#43

Post by fiddlerwoaroof »

cyphar wrote: post #22
beefsack wrote: post #17

ripgrep[1] is functionally incredibly similar to grep and ag, but is significantly faster[2] and supports a wider range of character encodings. In its short lifetime it has already become the default search tool for VSCode. I've switched to using it as my daily driver for text search and am incredibly happy with it. [1]: https://github.com/BurntSushi/ripgrep [2]: http://blog.burntsushi.net/ripgrep/ Edit: I confused a…

awk is a full programming language, and most of the times that I'm doing awk scripting I have to use things like associative arrays and arithmetic. Pulling out a field from a line is what most people use awk for, but it's honestly the least interesting part of awk. In fact, if cut supported regular expressions for specifying the field and record separators people wouldn't be using awk for that purpose (because that's…

The main reason I use awk 90% of the time is that its field parsing algorithm does "the right thing" in most cases (i.e. divide fields by 1 or more whitespace characters) without a lot of boilerplate, so it's really easy to throw into a pipeline.

Re: Skip grep, use awk

#44

Post by vidoc »

jsmthrowaway wrote: post #42
CaptSpify wrote: post #31

Earlier quoted context omitted.

I would just recommend pkill: https://en.wikipedia.org/wiki/Pkill

pgrep is also useful if you're not trying to kill. They're both from Solaris, and if you have one, you have the other. pgrep | xargs can also be very useful on occasion, depending on what you're doing (mainly debugging).

> xargs can also be very useful on occasion

yes, and xargs has a particularly useful argument -P that allows to do stuff in parallel and probably deserves more love :P

Re: Skip grep, use awk

#45

Post by cyphar »

fiddlerwoaroof wrote: post #43
cyphar wrote: post #22

Earlier quoted context omitted.

awk is a full programming language, and most of the times that I'm doing awk scripting I have to use things like associative arrays and arithmetic. Pulling out a field from a line is what most people use awk for, but it's honestly the least interesting part of awk. In fact, if cut supported regular expressions for specifying the field and record separators people wouldn't be using awk for that purpose (because that's…

The main reason I use awk 90% of the time is that its field parsing algorithm does "the right thing" in most cases (i.e. divide fields by 1 or more whitespace characters) without a lot of boilerplate, so it's really easy to throw into a pipeline.

That's what I was referencing when I said

> Pulling out a field from a line is what most people use awk for, but it's honestly the least interesting part of awk. In fact, if cut supported regular expressions for specifying the field and record separators people wouldn't be using awk for that purpose (because that's all that $n does).

There's nothing magical about awk's default FS. It's literally just /\s+/. If cut's -d was slightly more clever you wouldn't need to use awk.

Re: Skip grep, use awk

#46

Post by kotokz »

fiddlerwoaroof wrote: post #43
cyphar wrote: post #22

Earlier quoted context omitted.

awk is a full programming language, and most of the times that I'm doing awk scripting I have to use things like associative arrays and arithmetic. Pulling out a field from a line is what most people use awk for, but it's honestly the least interesting part of awk. In fact, if cut supported regular expressions for specifying the field and record separators people wouldn't be using awk for that purpose (because that's…

The main reason I use awk 90% of the time is that its field parsing algorithm does "the right thing" in most cases (i.e. divide fields by 1 or more whitespace characters) without a lot of boilerplate, so it's really easy to throw into a pipeline.

perl can do the same with -alne parms. but indeed awk has been widely used due to having this by default.

Re: Skip grep, use awk

#47

Post by bluetomcat »

AWK is the general-purpose programmatic filter and reporting tool in the Unix pipeline. Sed, grep and cut are specializations for specific use cases whose implementations might have better performance. Perl and Python are probably too general-purpose for writing compact one-liners in a pipeline.

Re: Skip grep, use awk

#49

Post by dredmorbius »

The reason my one-liners often end up with both grep and awk pipes is because that's how they're built: run some command, grep for content of interest, throw awk at it to manipulate output. If this is a throwaway, the exploration flexibility is more important than the command cleanness, and there are some awk idioms (inverted match, case-insensitive, match count, first match only) which are easier to code than their awk equivalents.

Sure, if I'm going to commit that to a script or shell function, I'll consider going back to refactor it and clean it up. But for quick-and-dirty exploration and prototyping, a long shell pipe is often the best modular way to compose a tool.

Protip: M-X-E will call up your one-liner into an editor session, from which you can save it directly to a permanent file. The number of locally-written tools which have originated in this fashion is ... probably embarassing to admit.

Re: Skip grep, use awk

#50

Post by bewuethr »

cyphar wrote: post #45
fiddlerwoaroof wrote: post #43

Earlier quoted context omitted.

The main reason I use awk 90% of the time is that its field parsing algorithm does "the right thing" in most cases (i.e. divide fields by 1 or more whitespace characters) without a lot of boilerplate, so it's really easy to throw into a pipeline.

That's what I was referencing when I said > Pulling out a field from a line is what most people use awk for, but it's honestly the least interesting part of awk. In fact, if cut supported regular expressions for specifying the field and record separators people wouldn't be using awk for that purpose (because that's all that $n does). There's nothing magical about awk's default FS. It's literally just /\s+/. If cut's…

The default FS throws away leading blanks, though, which doesn't happen if you set it explicitly to \s+, so a tiny little bit of magic does go on after all.
Post reply on HN