Parsing JSON in Forty Lines of Awk • Hacker News Board

Re: Parsing JSON in Forty Lines of Awk

#21

Post by latexr » Sat, Jun 28, 2025, 9:48 PM UTC

> Another option is to use Python, which is ubiquitous enough that it can be expected to be installed on virtually every machine

Not on macOS. You can do it easily by just invoking /usr/bin/python3, but you’ll get a (short and non-threatening) dialog to install the Xcode Command-Line Developer Tools. For macOS environments, JavaScript for Automation (i.e. JXA, i.e /usr/bin/osascript -l JavaScript) is a better choice.

However, since Sequoia, jq is now installed by default on macOS.

Re: Parsing JSON in Forty Lines of Awk

#22

Post by fragmede » Sat, Jun 28, 2025, 10:05 PM UTC

shawn_w wrote: post #16

chaps wrote: post #13

Earlier quoted context omitted.

Oh dang, that's good.

As long as you don't mind the extra space in the middle.

if you do:

    sed 's/  / /g'

Re: Parsing JSON in Forty Lines of Awk

#23

Post by chaps » Sat, Jun 28, 2025, 10:35 PM UTC

shawn_w wrote: post #16

chaps wrote: post #13

Earlier quoted context omitted.

Oh dang, that's good.

As long as you don't mind the extra space in the middle.

Often times I don't! Entirely depends on what I'm doing. #1 thing off the top of my head is to remove That One Column that's a bajillion characters long that makes exploratory analysis difficult.

Re: Parsing JSON in Forty Lines of Awk

#24

Post by saghm » Sat, Jun 28, 2025, 10:36 PM UTC

librasteve wrote: post #18

SoftTalker wrote: post #7

Earlier quoted context omitted.

> awk really shoots itself so much with its lack of features that it so desperately needs Whence perl.

or raku https://github.com/moritz/json/blob/master/lib/JSON/Tiny/Gra...

I suspect the rationale for Perl is that most Linux systems will probably have it installed already. Installing something you're familiar with is great when you can, but I'm guessing the awk script linked to here was picked more for its ubiquity than elegance.

Re: Parsing JSON in Forty Lines of Awk

#25

Post by saghm » Sat, Jun 28, 2025, 10:37 PM UTC

shawn_w wrote: post #16

chaps wrote: post #13

Earlier quoted context omitted.

Oh dang, that's good.

As long as you don't mind the extra space in the middle.

I wonder if it's possible to encode the backspace character in the replacement string?

Re: Parsing JSON in Forty Lines of Awk

#26

Post by lucb1e » Sat, Jun 28, 2025, 10:55 PM UTC

fragmede wrote: post #22

shawn_w wrote: post #16

Earlier quoted context omitted.

As long as you don't mind the extra space in the middle.

if you do: sed 's/ / /g'

Or sticking with awk, I have this bash alias to remove excess whitespace that is just:

    awk '{$1=$1};1'

Re: Parsing JSON in Forty Lines of Awk

#27

Post by chaps » Sat, Jun 28, 2025, 11:06 PM UTC

saghm wrote: post #24

librasteve wrote: post #18

Earlier quoted context omitted.

or raku https://github.com/moritz/json/blob/master/lib/JSON/Tiny/Gra...

I suspect the rationale for Perl is that most Linux systems will probably have it installed already. Installing something you're familiar with is great when you can, but I'm guessing the awk script linked to here was picked more for its ubiquity than elegance.

Kinda, but not really. Of the infrastructures I've worked on, not a single one has been consistent in installing perl on 100% of hosts. The ones that get close are usually like that because one high up person really, really likes perl. And they send a lot of angry emails about perl not being installed.

Within infrastructures where perl is installed on 95% of hosts, that 5% really bites you in the ass and leads to infrastructure rot very quickly. You're kinda stuck writing and maintaining two separate scripts to do the same thing.

Re: Parsing JSON in Forty Lines of Awk

#28

Post by SoftTalker » Sat, Jun 28, 2025, 11:32 PM UTC

chaps wrote: post #27

saghm wrote: post #24

Earlier quoted context omitted.

I suspect the rationale for Perl is that most Linux systems will probably have it installed already. Installing something you're familiar with is great when you can, but I'm guessing the awk script linked to here was picked more for its ubiquity than elegance.

Kinda, but not really. Of the infrastructures I've worked on, not a single one has been consistent in installing perl on 100% of hosts. The ones that get close are usually like that because one high up person really, really likes perl. And they send a lot of angry emails about perl not being installed. Within infrastructures where perl is installed on 95% of hosts, that 5% really bites you in the ass and leads to inf…

Same with Python. It's mostly available, but sometimes not.

With Perl, I find that a base installation is almost always available, but many packages might not be.

Re: Parsing JSON in Forty Lines of Awk

#29

Post by poke646 » Sat, Jun 28, 2025, 11:35 PM UTC

jcynix wrote: post #15

ptspts wrote: post #14

Earlier quoted context omitted.

This is much safer: xargs -d '\n' rm -f --

Sure, but my example was just that and I actually use /identical$/ as the pattern. Sorry for the typo. And I use this "historic" one liner only when I know about the contents of both directories. As soon as I need a "safer" solution I use a Perl script and pattern matching, as I said.

In this case the Perl one-liner would be conceptually identical, the same length, but more performant (no calling out to rm):

   diff -rs a/ b/ | perl -ane '/identical$/ && unlink $F[3]'

Re: Parsing JSON in Forty Lines of Awk

#30

Post by ZoomZoomZoom » Sat, Jun 28, 2025, 11:39 PM UTC

saghm wrote: post #25

shawn_w wrote: post #16

Earlier quoted context omitted.

As long as you don't mind the extra space in the middle.

I wonder if it's possible to encode the backspace character in the replacement string?

No problem, there's \b.

  echo "one two three four five" | awk '{$3="\b"; print}'

Inserts the backspace character (^H), which you can then remove with [global] substitution:

  awk '{$3="\b"; sub(/\32\b/, ""); print}'

You can, of course, use an arbitrary sentinel value for the field to be deleted. Should work in gawk and BWK awk.