Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Extracting Objects Recursively with Jq

til.simonwillison.net

Post reply on HN
Print view View original

71–76 of 76 posts

Re: Extracting Objects Recursively with Jq

#71

Post by 0xbadcafebee »

As a systems engineer working with containers, while I find jq really annoying to use, I much prefer it as a dependency to Python or Perl. Not only is it smaller, it's easier to install in weird environments.

Say you need to jump into a running container that isn't running as root, doesn't have sudo/su, and doesn't even have packaging tools. The only practical way to quickly install a debugging tool is either 1) volume-mount a file, 2) copy in a file, or 3) curl/wget. So a small(ish) static binary is the best thing you can hope for. I've actually built a dozen static versions of common debugging tools just to make this easier.

If I had more free time I would port jq's semantics to BusyBox and gain that much more utility out of that lovely little environment.

Re: Extracting Objects Recursively with Jq

#73

Post by Niksko »

psanford wrote: post #4

JQ is often frustrating when you want to do something non-trivial but you can't figure out how and the documentation is of little help. I think JQ could really benefit from having a classic programming language style "book", like "The AWK Programming Language". JQ is fundamentally a functional programming language with semantics that are not obvious reading its current docs.

Agreed. It's usually just a bunch of trial and error for me, or trying to phrase my problem in a way that matches stack overflow questions.

And the docs for yq (the jq for YAML) are 100 times worse. The documentation feels like it was written by aliens. Even incredibly simple things are impossible for me to find in the yq docs.

Re: Extracting Objects Recursively with Jq

#74

Post by xg15 »

Instead of using recurse(), you can also define a function in jq and call that one recursively. E.g., the example from the OP could also be written as:

  def my_func:
    del(.children),
    (.children[] | my_func)
  ;
  [my_func]
(The function first emits its input object without the "children" field, then emits each element of the "children" array separately, after passing each through a recursive call of the function.

The ; char ends the function definition.

Finally, the function is called once on the top-level input and all emitted results are collected into an array.)

This is more verbose than recurse() but may sometimes be easier to understand if you want to know what is going on.

Re: Extracting Objects Recursively with Jq

#75

Post by psanford »

psacawa wrote: post #56
psanford wrote: post #4

JQ is often frustrating when you want to do something non-trivial but you can't figure out how and the documentation is of little help. I think JQ could really benefit from having a classic programming language style "book", like "The AWK Programming Language". JQ is fundamentally a functional programming language with semantics that are not obvious reading its current docs.

This exists. At least, a detailed manual exists. It's the jq wiki on github [1]. Lots of detailed information/recipes. [1]: https://github.com/stedolan/jq/wiki

Whoa, I didn't know this existed. This does seem to address a lot of what I want. I wish that were more prominently linked to from https://stedolan.github.io/jq/.

Re: Extracting Objects Recursively with Jq

#76

Post by ossusermivami »

kbd wrote: post #62
hoshsadiq wrote: post #15

I tend to use jq a lot. As others have said, sometimes jq can be hard to grasp. Often it requires multiple attempts to get the correct answer. To make it a little easier for me, I've written a helper function[0] that combines it with fzf[1] to run jq as a REPL on any json. It allows to incrementally alter your DSL without having to continually call jq. This is similar to jid/jiq but a little more powerful. It include…

Just sharing my take on that interactive jq (or anything else) repl: https://github.com/kbd/setup/blob/master/HOME/bin/fzr It's just an fzf wrapper that sets up temporary files and so on. It works really well; it's amazing all the things one can use fzf for.

This is nice, it uses latest/modern python3 features, thanks, I needed some use case to see how those feature are useful in real world...
Post reply on HN