Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Zq: An easier and faster alternative to jq

brimdata.io

Post reply on HN
Print view View original

171–180 of 237 posts

Re: Zq: An easier and faster alternative to jq

#171

Post by dilap »

psacawa wrote: post #6

Since no one seems to know about it, jq is described in great detail on the github wiki page [0]. That flattens the learning curve a lot. It's not as arcane as it seems. The touted claim that is fundamentally stateless is not true. jq is also stateful in the sense that it has variables. If you want, you can write regular procedural code this way. Some examples [1] The real problem of jq is that it is currently lackin…

From that page:

> The jq documentation is written in a style that hides a lot of important detail because the hope is that the language feels intuitive.

Yeah, not so much boys! Also, that disclaimer should really be at the top of the manual, with a link to the wiki, rather than vice-versa, as it is now.

The wiki is like secret information -- "oh, hey, here's the page that actually tells you how it works!"

Re: Zq: An easier and faster alternative to jq

#172

Post by nixpulvis »

marmada wrote: post #158

I see a lot of JQ experts on this thread, so I'll bite the bullet here as a novice. The purpose of life is not to know JQ. I just want to process the JSON so I can move on and do whatever is actually important. Ideally, I'd just be able to tell GPT-codex to do what I want to do to the JSON in English. We're not there yet, but in the meantime if there's another tool that allows me to know less in exchange for doing mo…

No, not ideally.

English descriptions will never be completely unambiguous and unique keys into a JSON data structure. There is a very good reason programming languages (and other forms of languages) exist.

Re: Zq: An easier and faster alternative to jq

#173

Post by ar_lan »

Beltalowda wrote: post #57
psacawa wrote: post #6

Since no one seems to know about it, jq is described in great detail on the github wiki page [0]. That flattens the learning curve a lot. It's not as arcane as it seems. The touted claim that is fundamentally stateless is not true. jq is also stateful in the sense that it has variables. If you want, you can write regular procedural code this way. Some examples [1] The real problem of jq is that it is currently lackin…

> It's not as arcane as it seems. The issue with jq is that I use it maybe once a month, or even less. The syntax is "arcane enough" that I keep forgetting how to use it because I use it so sporadically. In comparison awk – which I also don't use that often – has a much easier syntax that I can mostly remember. Not entirely convinced by the zq syntax either though; it also seems "arcane enough" that I would keep forg…

This is ironic - I use `awk` so infrequently, I have no idea how to use it without reading its man page or using Google. But I use `jq` often and find it simple.

Re: Zq: An easier and faster alternative to jq

#174

Post by ptx »

sandyarmstrong wrote: post #170
ptx wrote: post #160

Earlier quoted context omitted.

Ah, yes, my mistake. Although PowerShell sends its own telemetry, the additional telemetry from the .NET platform is only sent when you use the dotnet command [1] and, as a special case, not when you very carefully invoke it only "in the following format: dotnet [path-to-app].dll" and never e.g. "dotnet help". However, presumably PowerShell requires at least the .NET Runtime if not the .NET SDK, doesn't it? The docs…

> However, presumably PowerShell requires at least the .NET Runtime if not the .NET SDK, doesn't it? Nope, these days .NET programs (like PowerShell) bundle the runtime. But even if they did a lighter distribution that depended on the runtime already being installed, there would be no .NET telemetry sent. > Does running the recommended "dotnet --list-runtimes" command send telemetry, like most of the commands? This i…

So the "dotnet" command is only in the SDK, not in the separately downloadable Runtime? Does the Runtime have some other command to launch an executable?

Edit: Actually, the ".NET Runtime 6.0.4" [1] (not the SDK) definitely has a "dotnet" command included. Presumably with the telemetry?

[1] https://dotnet.microsoft.com/en-us/download/dotnet/6.0

Re: Zq: An easier and faster alternative to jq

#175

Post by boyter »

marmada wrote: post #158

I see a lot of JQ experts on this thread, so I'll bite the bullet here as a novice. The purpose of life is not to know JQ. I just want to process the JSON so I can move on and do whatever is actually important. Ideally, I'd just be able to tell GPT-codex to do what I want to do to the JSON in English. We're not there yet, but in the meantime if there's another tool that allows me to know less in exchange for doing mo…

Same boat here. I ended up finding gron https://github.com/tomnomnom/gron which resolved that issue for me. Now I don't have to look up how to use jq each time I want to quickly find something in some JSON.

Re: Zq: An easier and faster alternative to jq

#176

Post by adamgordonbell »

xg15 wrote: post #162

A bit OT: The post links to the tutorial "An Introduction to JQ" at [1]. Somewhere inside the tutorial, array operators are introduced like this: > jq lets you select the whole array [], a specific element [3], or ranges [2:5] and combine these with the object index if needed. This is not supposed to be criticism on this particular tutorial (I've seen this kind of description quite often), but I could imagine this to…

Oh, I wrote that. I think I get what you mean. There are two different things, and they aren't being delineated. How would you explain it?

I don't know how jq works internally and in my mental model [] maps into the json array and also can wrap things back into an array. So that [.[]] unwraps and then rewraps a JSON array, sort of like how [.[].title] is the same as map(.title).

Re: Zq: An easier and faster alternative to jq

#177

Post by [deleted] »

marmada wrote: post #158

I see a lot of JQ experts on this thread, so I'll bite the bullet here as a novice. The purpose of life is not to know JQ. I just want to process the JSON so I can move on and do whatever is actually important. Ideally, I'd just be able to tell GPT-codex to do what I want to do to the JSON in English. We're not there yet, but in the meantime if there's another tool that allows me to know less in exchange for doing mo…

[deleted]

Re: Zq: An easier and faster alternative to jq

#178

Post by sfink »

The thing that I find myself wanting, which is lacking in both jq and zq afaik, is interactive exploration. I want to move around in a large JSON file, narrow my context to the portion I'm interested in, and do specialized queries and transformations on just the data I care about.

I wrote a tool to do this -- https://github.com/hotsphink/sfink-tools/blob/master/bin/jso... -- but I do not recommend it to anyone other than as perhaps a source of inspiration. It's slow and buggy, the syntax is cryptic and just matches whatever I came up with when I had a new need, etc. It probably wouldn't exist if I had heard of jq sooner.

But for what it does, it's awesome. I can do things like:

  % json somefile.json
  > ls
    0/
    1/
    2/
  > cd 0
  > ls
    info/
    files/
    timings/
    version
  > cat version
  1.2b
  > cat timings/*/mean
  timings/firstPaint/mean = 51
  timings/loadEventEnd/mean = 103
  timings/timeToContentfulPaint/mean = 68
  timings/timeToDomContentFlushed/mean = 67
  timings/timeToFirstInteractive/mean = 658
  timings/ttfb/mean = 6
There are commands for searching, modifying data, aggregating, etc., but those would be better done in a more principled, full-featured syntax like jq's.

I see ijq, and it looks really nice. But it doesn't have the context and restriction of focus that I'm looking for.

Re: Zq: An easier and faster alternative to jq

#179

Post by ggm »

sfink wrote: post #178

The thing that I find myself wanting, which is lacking in both jq and zq afaik, is interactive exploration. I want to move around in a large JSON file, narrow my context to the portion I'm interested in, and do specialized queries and transformations on just the data I care about. I wrote a tool to do this -- https://github.com/hotsphink/sfink-tools/blob/master/bin/jso... -- but I do not recommend it to anyone other…

This is almost exactly how I think about the problem of deciding how to deep-key to a specific field of a nested json structure.

If you can emit the syntactic form as a Python or perl ref, or a jq array ref, then I could use your tool to find the structure and the other ones to stream.

Great example! Thanks for posting this.

Re: Zq: An easier and faster alternative to jq

#180

Post by phil294 »

marmada wrote: post #158

I see a lot of JQ experts on this thread, so I'll bite the bullet here as a novice. The purpose of life is not to know JQ. I just want to process the JSON so I can move on and do whatever is actually important. Ideally, I'd just be able to tell GPT-codex to do what I want to do to the JSON in English. We're not there yet, but in the meantime if there's another tool that allows me to know less in exchange for doing mo…

This is one of the purposes I think Deno should have been built for: Use JavaScript for oneliners in the command line. We had

    ... | deno xeval '...stdin processing code using special var $'
which was close to xargs in terms of conciseness. Unfortunately, it was removed as being considered "too niche" [1].

[1] https://github.com/denoland/deno/issues/3230

Post reply on HN