Live data from Hacker News

FX: An interactive alternative to jq to process JSON

github.com

51–60 of 65 posts

Re: FX: An interactive alternative to jq to process JSON

#51
post #48

Earlier quoted context omitted.

Not sure I understand. I mean using the interactive mode to figure out the right query ("[].mumble.whatever...") and then being able to save the text of the query that you figured out by pointing and clicking. Like a graphical SQL query builder allows you to do.

You can do this with `jid -q`: https://github.com/simeji/jid (Its interactivity is keyboard- rather than mouse-based.)

Oh, that's great...outputs a jq-compatible query. Thanks!

Re: FX: An interactive alternative to jq to process JSON

#52
post #29
post #9

JQ syntax feels too unusual, doesn't resemble known code, gives me the feeling of looking into cryptic Perl or regex, could never remember the simplest things. For example how would you take key k1 from a list of dicts [{k1: v1, k2: v2}, {k1: v3}]?

> For example how would you take key k1 from a list of dicts [{k1: v1, k2: v2}, {k1: v3}]? Do you mean something like: .[].k1 Give it a try. https://jqplay.org/ jq does have a learning curve, but just like any query language, including SQL, first you need to learn the basics of the query language in order to get things to work. In this case: * you know that .[] iterates over objects, so you use it to unpack the root…

Well for comparison, I did that learning curve process with SQL and I was able to understand it. But I did the same learning curve process with JQ and I still don’t understand it.

Re: FX: An interactive alternative to jq to process JSON

#55

This simply evals the argument. Not an alternative to jq. fx “code to eval”

This doesn't look to me something that can be evaluated by node

  fx .comments[].authors[].names
You might want to take a look at this post[1] to see everything fx can do.

[1] https://medium.com/@antonmedv/discover-how-to-use-fx-effecti...

Re: FX: An interactive alternative to jq to process JSON

#56
For those who are comfortable with python, I created Jello[0], which works like jq but uses python syntax.

I also created Jellex[1], which is a TUI built on Jello to assist with building the python queries.

Jello gives you the power of python but without all of the boilerplate, so it’s nicer to use in Bash scripts.

[0] https://github.com/kellyjonbrazil/jello

[1] https://github.com/kellyjonbrazil/jellex

Re: FX: An interactive alternative to jq to process JSON

#57

Looks really nice. I wish it handled yaml as well.

I like to have something like this in my shell:

    y2j () {
        ruby -r json -r yaml -e 'puts JSON.dump(YAML.load(STDIN))'
    }
Makes it easy to use json tooling for yaml, although of course it flattens out anchors etc.

Re: FX: An interactive alternative to jq to process JSON

#58
post #19

Quoted post unavailable.

On my machine jq depends on oniguruma. Both node and onigurama should be easy to install from your distro package manager.

It's because Homebrew (if you're on macOS) decided not to compile it statically.

Re: FX: An interactive alternative to jq to process JSON

#59
post #9

JQ syntax feels too unusual, doesn't resemble known code, gives me the feeling of looking into cryptic Perl or regex, could never remember the simplest things. For example how would you take key k1 from a list of dicts [{k1: v1, k2: v2}, {k1: v3}]?

> gives me the feeling of looking into cryptic Perl or regex

Dunno if you'll see this given how many replies you already got, but rather than just dumping "how do you do that" here's a realization I had a while ago that made it way easier to understand:

jq's language is a series of filters/transformers more akin to bash pipes on a stream of data than anything else.

For example, just "." selects out the current object (and is needed to match the "root" at the start of the query), and jq pretty-prints the results (when to a terminal):

  $ echo '[{"k1": "v1", "k2": "v2"}, {"k1": "v3"}]'
  [{"k1": "v1", "k2": "v2"}, {"k1": "v3"}]
  $ echo '[{"k1": "v1", "k2": "v2"}, {"k1": "v3"}]' | jq '.'
  [
    {
      "k1": "v1",
      "k2": "v2"
    },
    {
      "k1": "v3"
    }
  ]
There's only 1 matching element here, the outermost array. We want to go one deeper, so use "[]" to unwrap/flatten it:

  $ echo '[{"k1": "v1", "k2": "v2"}, {"k1": "v3"}]' | jq '.[]'
  {
    "k1": "v1",
    "k2": "v2"
  }
  {
    "k1": "v3"
  }
jq is now iterating over 2 objects, so the next filter is the one where you select out the key you want. This can be done in two different ways for this example (per sibling replies):

  $ echo '[{"k1": "v1", "k2": "v2"}, {"k1": "v3"}]' | jq '.[].k1'
  "v1"
  "v3"
  $ echo '[{"k1": "v1", "k2": "v2"}, {"k1": "v3"}]' | jq '.[] | .k1'
  "v1"
  "v3"
Note how I broke these up: The atoms are ".", "[]", and ".k1" - ".[]" isn't one of them, despite what it may look like at first glance when compared to ".k1". Some additional examples to show how these combine:

The "unwrap/flatten" [] can be used multiple times when nested arrays are involved, with or without the pipe syntax, but only works on arrays. It errors if given something else:

  $ echo '[[1,2,3],[4,[5,6]]]' | jq '.[]'
  [
    1,
    2,
    3
  ]
  [
    4,
    [
      5,
      6
    ]
  ]
  $ echo '[[1,2,3],[4,[5,6]]]' | jq '.[][]'
  1
  2
  3
  4
  [
    5,
    6
  ]
  $ echo '[[1,2,3],[4,[5,6]]]' | jq '.[][][]'
  jq: error (at :1): Cannot iterate over number (1)

  $ echo '[[1,2,3],[4,[5,6]]]' | jq '.[] | .[]'
  1
  2
  3
  4
  [
    5,
    6
  ]
  $ echo '[[1,2,3],[4,[5,6]]]' | jq '.[] | .[] | .[]'
  jq: error (at :1): Cannot iterate over number (1)
Also notice how the "." is needed after the pipes; these are separate filters/transformations being chained together, so as a new rule it needs the same "." as with the first one.

Re: FX: An interactive alternative to jq to process JSON

#60
post #29
post #9

JQ syntax feels too unusual, doesn't resemble known code, gives me the feeling of looking into cryptic Perl or regex, could never remember the simplest things. For example how would you take key k1 from a list of dicts [{k1: v1, k2: v2}, {k1: v3}]?

> For example how would you take key k1 from a list of dicts [{k1: v1, k2: v2}, {k1: v3}]? Do you mean something like: .[].k1 Give it a try. https://jqplay.org/ jq does have a learning curve, but just like any query language, including SQL, first you need to learn the basics of the query language in order to get things to work. In this case: * you know that .[] iterates over objects, so you use it to unpack the root…

> jq does have a learning curve, but just like any query language, including SQL, first you need to learn the basics of the query language in order to get things to work.

SQL is based on solid mathematical theory, relational algebra. I personally learned that (and tuple relational calculus) in college before learning SQL, which made it easier. It helps making it coherant. Is there something like this for jq? Often when people invent languages that are not based on solid theory, they tend to lack coherence. This can make learning them difficult if you're someone that relies on your mental model of how things "should" work, like I am.

Post reply on HN