21–30 of 65 posts
Quoted post unavailable.
On my machine jq depends on oniguruma. Both node and onigurama should be easy to install from your distro package manager.
Apparently, the author wrote this tool because jid was struggling with a 7MB JSON file. See https://github.com/simeji/jid/issues/66#issuecomment-4436718...
A generic (partial) solution to this type of thing is just to sample a number of lines from the large input, and do the investigation on that. shuf -n 1000 file This is part of coreutils. There's also jiq, which is a clone of jid (mentioned elsewhere) but with jq syntax
JQ syntax feels too unusual, doesn't resemble known code, gives me the feeling of looking into cryptic Perl or regex, could never remember the simplest things. For example how would you take key k1 from a list of dicts [{k1: v1, k2: v2}, {k1: v3}]?
I agree that jq's query language is very obtuse and probably my biggest barrier towards learning it. I have found great mileage using gron [1], which is very different from jq, but its goal is to promote exploration of a JSON file through common unix tools such as awk and grep. 1: https://github.com/tomnomnom/gron
`gron` is great but doesn't seem to handle some (extreme-ish) situations that `jq` can, e.g. the json output from the fastnbt-tools. You either get a `token too long` error using `gron -s` because the input is too long (it's 90MB, that's fair) or you get only one set of outputs per key (iyswim) because they get overlapped in memory.
Earlier quoted context omitted.
A generic (partial) solution to this type of thing is just to sample a number of lines from the large input, and do the investigation on that. shuf -n 1000 file This is part of coreutils. There's also jiq, which is a clone of jid (mentioned elsewhere) but with jq syntax
That would only work if it's line separated JSON though. If you cut off the first 1000 lines of a big JSON file it will be invalid.
jq —-compact-output '.' | head -10 | foo
jq also has --stream for handling large inputs.
JQ syntax feels too unusual, doesn't resemble known code, gives me the feeling of looking into cryptic Perl or regex, could never remember the simplest things. For example how would you take key k1 from a list of dicts [{k1: v1, k2: v2}, {k1: v3}]?
Had the same experience. That's why I've written jql[0], which puts a uniform lispy spin on CLI JSON processing. I now use it almost exclusively instead of jq. Check it out if you're looking for alternatives. And by the way, you can achieve live preview with any of these CLI tools by using fzf. This is the snippet for jql for example: `echo '' | fzf --print-query --preview-window wrap --preview 'cat test.json | jql {…
JQ syntax feels too unusual, doesn't resemble known code, gives me the feeling of looking into cryptic Perl or regex, could never remember the simplest things. For example how would you take key k1 from a list of dicts [{k1: v1, k2: v2}, {k1: v3}]?
'.[].k1'Earlier quoted context omitted.
I agree that jq's query language is very obtuse and probably my biggest barrier towards learning it. I have found great mileage using gron [1], which is very different from jq, but its goal is to promote exploration of a JSON file through common unix tools such as awk and grep. 1: https://github.com/tomnomnom/gron
> I have found great mileage using gron `gron` is great but doesn't seem to handle some (extreme-ish) situations that `jq` can, e.g. the json output from the fastnbt-tools. You either get a `token too long` error using `gron -s` because the input is too long (it's 90MB, that's fair) or you get only one set of outputs per key (iyswim) because they get overlapped in memory.
That sounds like a major bug. So it will silently skip data that you wanted?
JQ syntax feels too unusual, doesn't resemble known code, gives me the feeling of looking into cryptic Perl or regex, could never remember the simplest things. For example how would you take key k1 from a list of dicts [{k1: v1, k2: v2}, {k1: v3}]?
╰─$ echo '[{"k1": "v1", "k2": "v2"}, {"k1": "v3"}]' | jq -r '.[] | .k1'
v1
v3
1. parse a json value from stdin and set it as the initial result
2. for each function, apply the function to the result, and set the output as the result for the next function.
3. The final result is pretty printed on stdout.
JQ syntax feels too unusual, doesn't resemble known code, gives me the feeling of looking into cryptic Perl or regex, could never remember the simplest things. For example how would you take key k1 from a list of dicts [{k1: v1, k2: v2}, {k1: v3}]?
Do you mean something like:
.[].k1
jq does have a learning curve, but just like any query language, including SQL, first you need to learn the basics of the query language in order to get things to work.
In this case:
* you know that .[] iterates over objects, so you use it to unpack the root array,
* you know you get a stream of objects, thus from those you use the .k1 filter to get the values of each k1 key.
Here's jq's manual on basic filters: https://stedolan.github.io/jq/manual/#Basicfilters
After you get jq to filter out what you want, you can work on getting it to output results in whatever format you wish.
JQ syntax feels too unusual, doesn't resemble known code, gives me the feeling of looking into cryptic Perl or regex, could never remember the simplest things. For example how would you take key k1 from a list of dicts [{k1: v1, k2: v2}, {k1: v3}]?
My approach to the example would be to use `.[] | .k1` which I think does what you want, and like bash command line pipes, you can build up to it semi-interactively.
The bits I struggle with JQ often involve irregular json, where a value might be missing, or null, or a list, not sure what the idiomatic way to deal with that is if there is one.