Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

JJ: JSON Stream Editor

github.com

Post reply on HN
Print view View original

21–30 of 50 posts

Re: JJ: JSON Stream Editor

#21

Post by maleldil »

Am I correct in understanding that this can only manipulate (get or set values) from a JSON path? That is, is it not a replacement for jq?

For example, I frequently use jq for queries like this:

    jq '.data | map(select(.age
Or this:

    jq '.data | map(.country) | sort[]' input.json | uniq -c
Is it possible to do something similar with this tool?

This is not a slight at jj. Even if it's more limited than jq, it's still of great value if it means it's faster or more ergonomic for a subset of cases. I'm just trying to understand how it fits in my toolbox.

Re: JJ: JSON Stream Editor

#22

Post by robertlagrant »

qhwudbebd wrote: post #10

Interesting! I tend to use gron to bring JSON into (and out of) the line-based bailiwick of sed and awk where I'm most comfortable, rather than a custom query language like jq that I'd use much more rarely. But I guess that's at the opposite extreme of (in)efficiency than both this and the original jq. There might be a nice 'edit just this path in-place in gron-style' recipe to be had out of jj/jq + gron together...

Just looked up gron - thanks. This looks useful.

Re: JJ: JSON Stream Editor

#23

Post by j1elo »

I'll take the chance to bring attention to the maintenance issues that 'jq' has been having in the last years [1]; there hasn't been a new release since 2018, which IMO wouldn't necessarily be a bad thing if not for the fact that the main branch has been collecting improvements and bug fixes [2] since then.

A group of motivated users are currently talking about what direction to take; a fork is being considered in order to unlock new development and bug fixes [3]. Maybe someone reading this is able and willing to join their efforts.

[1]: https://github.com/stedolan/jq/issues/2305

[2]: https://github.com/stedolan/jq/pull/1697

[3]: https://github.com/stedolan/jq/issues/2550

Re: JJ: JSON Stream Editor

#24

Post by capableweb »

j1elo wrote: post #23

I'll take the chance to bring attention to the maintenance issues that 'jq' has been having in the last years [1]; there hasn't been a new release since 2018, which IMO wouldn't necessarily be a bad thing if not for the fact that the main branch has been collecting improvements and bug fixes [2] since then. A group of motivated users are currently talking about what direction to take; a fork is being considered in or…

What exactly is missing/broken in jq right now which warrants a fork? I've been using jq daily for years, and I can't remember the last time I hit a bug (must have been many years ago) and I can't recall any features I felt been missing for the years I've been using it.

For me it's kind of done. It could be faster, but then I tend to program a solution myself instead, otherwise I feel like it's Done Enough.

Re: JJ: JSON Stream Editor

#25

Post by capableweb »

vmfunction wrote: post #11
asadm wrote: post #5

I wish this existed when I was trying to look at 20G of firebase database JSON dump.

that is what gets me, why did the file get to 20g? At that point just ship a SQLite file.

Does it matter why? Sometimes files gets big, and you don't control the generation or trying to change the generation is a bigger task than just dealing with a "big" (I'd argue 20GB isn't that big anyways) file with standard tools.

Re: JJ: JSON Stream Editor

#26

Post by Rygian »

This behaviour looks confusing to me:

$ echo '{"name":{"first":"Tom","middle":"null","last":"Smith"}}' | jj name.middle

null

$ echo '{"name":{"first":"Tom","last":"Smith"}}' | jj name.middle

null

It can be avoided with option '-r' which should be the default, but is not.

Re: JJ: JSON Stream Editor

#27

Post by AeroNotix »

lathiat wrote: post #7
pcthrowaway wrote: post #4

Earlier quoted context omitted.

I don't think I've ever been limited by jq's speed, but good to know there are alternatives if it ever becomes a bottleneck. Other than that I can't think of a reason to use this over jq; the query language is perhaps a bit more forgiving in some ways, but not as expressive as jq (and I've spent ~8 years getting pretty familiar with jq's quirks)

The limiting speed factor of jq for me is, by far, figuring out how to write the expression I need to parse a fairly small amount of data. I do a bunch of support analysis and often writing a one-liner to put into a shell script to extract some bit of JSON to re-use later in the script. Often this is going to be used only once by me or a customer to run some task. Followed closely by figuring out the path to the area…

emacs has a command to get the current path at point.

Re: JJ: JSON Stream Editor

#28

Post by goranmoomin »

capableweb wrote: post #24
j1elo wrote: post #23

I'll take the chance to bring attention to the maintenance issues that 'jq' has been having in the last years [1]; there hasn't been a new release since 2018, which IMO wouldn't necessarily be a bad thing if not for the fact that the main branch has been collecting improvements and bug fixes [2] since then. A group of motivated users are currently talking about what direction to take; a fork is being considered in or…

What exactly is missing/broken in jq right now which warrants a fork? I've been using jq daily for years, and I can't remember the last time I hit a bug (must have been many years ago) and I can't recall any features I felt been missing for the years I've been using it. For me it's kind of done. It could be faster, but then I tend to program a solution myself instead, otherwise I feel like it's Done Enough.

> What exactly is missing/broken in jq right now which warrants a fork

AFAIK there’s quite a few bug fixes and features that are accumulated on the unreleased main branch, or opened as PRs but never merged.

IIRC I hit one of the bugs while trying to check whether an input document is valid JSON.

I should try checking out what’s happening to the fork, I’ve never opened a PR or something but I’ve read the source while trying to understand the jq language conceptually, and I’d say it’s quite elegant :)

Re: JJ: JSON Stream Editor

#29

Post by bobnamob »

lathiat wrote: post #7
pcthrowaway wrote: post #4

Earlier quoted context omitted.

I don't think I've ever been limited by jq's speed, but good to know there are alternatives if it ever becomes a bottleneck. Other than that I can't think of a reason to use this over jq; the query language is perhaps a bit more forgiving in some ways, but not as expressive as jq (and I've spent ~8 years getting pretty familiar with jq's quirks)

The limiting speed factor of jq for me is, by far, figuring out how to write the expression I need to parse a fairly small amount of data. I do a bunch of support analysis and often writing a one-liner to put into a shell script to extract some bit of JSON to re-use later in the script. Often this is going to be used only once by me or a customer to run some task. Followed closely by figuring out the path to the area…

I’m far more likely to parse json into clojure repl session and go from there these days. Learning jq for the odd json manipulation I need to do seems like overkill

Re: JJ: JSON Stream Editor

#30

Post by zokier »

nigeltao wrote: post #8
BiteCode_dev wrote: post #2

For those wondering, the README states it's a lot faster than JQ, which may be the selling point.

jj is faster than jq. However, jsonptr is even faster and also runs in a self-imposed SECCOMP_MODE_STRICT sandbox (very secure; also implies no dynamically allocated memory). $ time cat citylots.json | jq -cM .features[10000].properties.LOT_NUM "091" real 0m4.844s $ time cat citylots.json | jj -r features.10000.properties.LOT_NUM "091" real 0m0.210s $ time cat citylots.json | jsonptr -q=/features/10000/properties/LOT…

Looks neat. One suggestion: add better build instructions on wuffs readme/getting started guide. I jumped in and tried to build it using the "build-all.sh" script that seemed convenient, but gave up (for now) after nth build failure due yet another missing dependency. It's extra painful because the build-all.sh is slow, so maybe also consider some proper build automation tool (seeing this is goog project, maybe bazel?)?
Post reply on HN