I wish this existed when I was trying to look at 20G of firebase database JSON dump.
that is what gets me, why did the file get to 20g? At that point just ship a SQLite file.
JJ: JSON Stream Editor
11–20 of 50 posts
Re: JJ: JSON Stream Editor
#12Interesting! I tend to use gron to bring JSON into (and out of) the line-based bailiwick of sed and awk where I'm most comfortable, rather than a custom query language like jq that I'd use much more rarely. But I guess that's at the opposite extreme of (in)efficiency than both this and the original jq. There might be a nice 'edit just this path in-place in gron-style' recipe to be had out of jj/jq + gron together...
Are there any gron-like tools for xml? I'm aware it's a harder problem (and an increasingly rare problem) but perhaps someone has tackled it nonetheless?
Re: JJ: JSON Stream Editor
#13I would like to see a comparison with jshon. Jshon is way faster than jq and for many years available in your distro repositories.
Cool, didn’t know about jshon, how’s the query language?
Re: JJ: JSON Stream Editor
#14Earlier quoted context omitted.
I don't think I've ever been limited by jq's speed, but good to know there are alternatives if it ever becomes a bottleneck. Other than that I can't think of a reason to use this over jq; the query language is perhaps a bit more forgiving in some ways, but not as expressive as jq (and I've spent ~8 years getting pretty familiar with jq's quirks)
The limiting speed factor of jq for me is, by far, figuring out how to write the expression I need to parse a fairly small amount of data. I do a bunch of support analysis and often writing a one-liner to put into a shell script to extract some bit of JSON to re-use later in the script. Often this is going to be used only once by me or a customer to run some task. Followed closely by figuring out the path to the area…
I agree that figuring out non-trivial jq expressions takes a lot of time, often accompanied with a consultation of the somewhat lacking docs, and some additional googling.
Nonetheless, it is pretty slow at processing data. For example, converting a 1 GB JSON array of objects to JSON Lines takes ages, if it works at all. Using the steaming features helps, but they are hard to comprehend. It gets memory consumption under control and doesn't take super long, but still way too long for such a trivial task IMO.
Re: JJ: JSON Stream Editor
#15Is this the SAX of JSON?
Re: JJ: JSON Stream Editor
#16I would like to see a comparison with jshon. Jshon is way faster than jq and for many years available in your distro repositories.
Cool, didn’t know about jshon, how’s the query language?
Almost non-existing. A couple of excerpts from man page:
{"a":1,"b":[true,false,null,"str"],"c":{"d":4,"e":5}}
jshon [actions] {"d":4,"e":5}
jshon -e c -e d -u -p -e e -u -> 4 5
Re: JJ: JSON Stream Editor
#17[deleted]
Re: JJ: JSON Stream Editor
#18Interesting! I tend to use gron to bring JSON into (and out of) the line-based bailiwick of sed and awk where I'm most comfortable, rather than a custom query language like jq that I'd use much more rarely. But I guess that's at the opposite extreme of (in)efficiency than both this and the original jq. There might be a nice 'edit just this path in-place in gron-style' recipe to be had out of jj/jq + gron together...
Are there any gron-like tools for xml? I'm aware it's a harder problem (and an increasingly rare problem) but perhaps someone has tackled it nonetheless?
xml2[1] turns xml into line-based output. and 2xml reverses.
Re: JJ: JSON Stream Editor
#19Earlier quoted context omitted.
I don't think I've ever been limited by jq's speed, but good to know there are alternatives if it ever becomes a bottleneck. Other than that I can't think of a reason to use this over jq; the query language is perhaps a bit more forgiving in some ways, but not as expressive as jq (and I've spent ~8 years getting pretty familiar with jq's quirks)
The limiting speed factor of jq for me is, by far, figuring out how to write the expression I need to parse a fairly small amount of data. I do a bunch of support analysis and often writing a one-liner to put into a shell script to extract some bit of JSON to re-use later in the script. Often this is going to be used only once by me or a customer to run some task. Followed closely by figuring out the path to the area…
> Switching to a GUI to browse the JSON that would let you copy the path to the current value would probably also help there
Try https://jless.io/ then.
Re: JJ: JSON Stream Editor
#20Earlier quoted context omitted.
I don't think I've ever been limited by jq's speed, but good to know there are alternatives if it ever becomes a bottleneck. Other than that I can't think of a reason to use this over jq; the query language is perhaps a bit more forgiving in some ways, but not as expressive as jq (and I've spent ~8 years getting pretty familiar with jq's quirks)
The limiting speed factor of jq for me is, by far, figuring out how to write the expression I need to parse a fairly small amount of data. I do a bunch of support analysis and often writing a one-liner to put into a shell script to extract some bit of JSON to re-use later in the script. Often this is going to be used only once by me or a customer to run some task. Followed closely by figuring out the path to the area…
> Switching to a GUI to browse the JSON that would let you copy the path to the current value would probably also help there
I use an app called OK JSON on the mac for this. Its okay.