Zq: An easier and faster alternative to jq

Post by dotopotoro » Tue, Apr 26, 2022, 9:53 PM UTC

marmada wrote: post #158

I see a lot of JQ experts on this thread, so I'll bite the bullet here as a novice. The purpose of life is not to know JQ. I just want to process the JSON so I can move on and do whatever is actually important. Ideally, I'd just be able to tell GPT-codex to do what I want to do to the JSON in English. We're not there yet, but in the meantime if there's another tool that allows me to know less in exchange for doing mo…

Post by thayne » Tue, Apr 26, 2022, 9:53 PM UTC

knome wrote: post #20

These guys must really hate functional programming. I can see where jq might confuse someone new to it, but their replacement is irregular, stateful, still difficult, and I don't even see variable binding or anything. jq requires you to understand that `hello|world` will run world for each hello, passing the world out values to either the next piped expression, the wrapping value-collecting list, or printing them to…

Post by lichtenberger » Tue, Apr 26, 2022, 9:59 PM UTC

sfink wrote: post #178

The thing that I find myself wanting, which is lacking in both jq and zq afaik, is interactive exploration. I want to move around in a large JSON file, narrow my context to the portion I'm interested in, and do specialized queries and transformations on just the data I care about. I wrote a tool to do this -- https://github.com/hotsphink/sfink-tools/blob/master/bin/jso... -- but I do not recommend it to anyone other…

    let $node := jn:doc('mycol.jn','mydoc.jn')=>fieldName[[1]]
    let $result := for $node-in-rev in jn:all-times($node)
                   return
                     if ((not(exists(jn:previous($node-in-rev))))
                          or (sdb:hash($node-in-rev) ne sdb:hash(jn:previous($node-in-rev)))) then
                       $node-in-rev
                     else
                       ()
    return [
      for $jsonItem in $result
      return { "node": $jsonItem, "revision": sdb:revision($jsonItem) }
    ]

    let $maxRevision := sdb:revision(jn:doc('mycol.jn','mydoc.jn'))
    let $result := for $i in (1 to $maxRevision)
                   return
                     if ($i > 1) then
                       jn:diff('mycol.jn','mydoc.jn',$i - 1, $i)
                     else
                       ()
    return [
      for $diff at $pos in $result
      return {"diffRev" || $pos || "toRev" || $pos + 1: jn:parse($diff)=>diffs}
    ]

    jn:open('mycol.jn','mydoc.jn',xs:dateTime('2022-03-01T00:00:00Z'))

    jn:doc('mycol.jn','mydoc.jn',5)

Post by lgas » Tue, Apr 26, 2022, 10:07 PM UTC

laurent123456 wrote: post #103

Beltalowda wrote: post #57

Earlier quoted context omitted.

> It's not as arcane as it seems. The issue with jq is that I use it maybe once a month, or even less. The syntax is "arcane enough" that I keep forgetting how to use it because I use it so sporadically. In comparison awk – which I also don't use that often – has a much easier syntax that I can mostly remember. Not entirely convinced by the zq syntax either though; it also seems "arcane enough" that I would keep forg…

I wonder if someone tried to use plain JS as a filtering language? It would be more verbose but it would be easy to remember. For example: [1,2,3] | js "out = 0; for (const n of this) out += n" That would print "6". `out` would be a special variable you write to to print the result, and `this` would be the input.

Post by Beltalowda » Tue, Apr 26, 2022, 10:27 PM UTC

dotancohen wrote: post #137

Beltalowda wrote: post #110

Earlier quoted context omitted.

I don't know; I wouldn't presume to tell you what you do or don't find arcane, but once I understood the somewhat unusual flow of awk ("for every line, check if the line matches this condition, and if it does run this block of code") I found it's quite easy to work with. It's "arcane" in the sense that it has an implicit loop and that it's a specialized language for a very limited class of problems, but I found that…

> an implicit loop As an occasional awk user, I'd love if you expand on this. Maybe it will help clear things up for me. You're not referring to the fact that awk operates on every line independently, are you?

    for line in readfile()
        for block in script:
            if block.match(line)
                run_block(block)
            end
        endfor
    endfor