Live data from Hacker News

Show HN: Catj – A new way to display JSON files

github.com

111–120 of 127 posts

Re: Show HN: Catj – A new way to display JSON files

#111
post #96

Earlier quoted context omitted.

That's awesome work. The only problem is that it does not properly handle keys which are not valid JS identifiers (like 1foo, @foo, foo-bar, etc.).

Well, there's this option without the blacklist: jq -r ' tostream | select(length > 1) | (.[0] | map("[" + @json + "]") | join("")) + " = " + (.[1] | @json) ' And this other option with the blacklist patterns: jq -r ' tostream | select(length > 1) | ( .[0] | map( if type == "number" or (tostring | test("[@-]|^[0-9]|^else$")) then "[" + @json + "]" else "." + . end ) | join("") ) + " = " + (.[1] | @json) ' (The blackl…

It occurs to me that a better way to blacklist would be something like:

  jq -r '
    tostream
    | select(length > 1)
    | (
      .[0] | map(
        if tostring | (
          test("^[A-Za-z$_][0-9A-Za-z$_]*$")
          and (
            . as $property
            | ["if", "else"] | all(. != $property) 
          )
        )
        then "." + .
        else "[" + @json + "]"
        end
      ) | join("")
    ) + " = " + (.[1] | @json)
  '
You whitelist against what the syntax allows for identifiers and then you blacklist reserved keywords. Writing it this way makes it easier to verify for correctness when comparing with the ECMAScript Specs. This is still a non-exhaustive blacklist and the whitelist regex lacks allowed unicode characters.

Re: Show HN: Catj – A new way to display JSON files

#112
post #7

this is not merely a display, it is really a sanitation of the brain-damaged json syntax.

I have never once heard this opinion of JSON. JSON is a breath of fresh air. I'm really struggling to imagine what sort of software you work on.

If you come from xml hell, then yes, json may even seem a reasonable option. But if you are used to the terse beauty of printf and scanf, then a json file looks like bad sarcasm.

Re: Show HN: Catj – A new way to display JSON files

#113
post #20

I was curious if this could be doable with jq, and apparently it is: jq -j ' [ [ paths(scalars) | map( if type == "number" then "[" + tostring + "]" else "." + . end ) | join("") ], [ .. | select(scalars) | @json ] ] | transpose | map(join(" = ") + "\n") | join("") ' EDIT: Got the string quoting and escaping. EDIT 2: For those who want to save this script, you can put just the jq code in an executable file with the s…

jq: error: syntax error, unexpected INVALID_CHARACTER, expecting $end (Unix shell quoting issues?) at , line 3: jq -j ' jq: 1 compile error This, uh, doesn't work for me on jq-1.5.1.

#!/usr/local/bin/jq -rf

tostream | select(length > 1) | ( .[0] | map( if type == "number" then "[" + tostring + "]" else "." + . end ) | join("") ) + " = " + (.[1] | @json)

Re: Show HN: Catj – A new way to display JSON files

#114
post #67

Earlier quoted context omitted.

macOS appears to support multiple args just fine. Which is why it annoys me that Shellcheck bitches about using more than one arg even though I'm writing a script for macOS specifically.

ShellCheck is right insofar as compatibility is concerned. You can only rely on the shebang supporting one argument. I'd personally just ignore that warning if I were writing for MacOS specifically, but you can configure ShellCheck to ignore certain errors that you don't care about[1]. [1] https://github.com/koalaman/shellcheck/wiki/Ignore

Yeah, but it's simpler just to move the additional args to `set` calls than to remember the syntax for disabling the directive. It's just irritating, and especially so because, for some reason, in VSCode it ends up highlighting literally the entire file as an error instead of just the shebang.

Re: Show HN: Catj – A new way to display JSON files

#115
post #72

Earlier quoted context omitted.

Appending | to each line and a last line . is a jq program that reproduces the original json. jq -r ' ( tostream | select(length > 1) | ( .[0] | map( if type == "number" then "[" + tostring + "]" else "." + . end ) | join("") ) + " = " + (.[1] | @json) + " |" ), "." '

That's insane. Here I was thinking about how much more challenging it would be to parse and reconstruct the object from jq, and you got the idea to take advantage of the syntax similarity to parse it as jq code itself. Nice. And so, that means the inverse of the jq code I posted would simply be: ( jq "$(sed 's/$/ |/;$a.')" As in: catj example.json \ | ( jq "$(sed 's/$/ |/;$a.')" original.json

Parsing is awkward in jq, but setpath(PATHS; VALUE) will create necessary structure. PATHS uses the array form, like ["movie", "cast", 5] not .movie.cast[5]. Since 1.5, jq has PCRE regex, so could remove ], and separate by . and [.

Re: Show HN: Catj – A new way to display JSON files

#116
post #75
post #74

Earlier quoted context omitted.

It does look like neither are needed if you pipe a file in jq, but `jq . file.json` requires the `.` and if you're pipeing into a pager, like less, you need both `.` and `-C` to get colored output (that was the case with the alias I had pulled up). I am using 1.5 and haven't looked to see if 1.6 changes this.

I see. I doubt that behavior has changed, then. `-C` would be required when piping because most of the time (with the exception of piping into less) when stdout is not a terminal, it doesn't make sense to include terminal color escape sequences. You'd end up with those codes in your files, and grep would be looking at them for matches, for example. `.` would be required when passing the file as an argument instead of…

`.` is still needed if I'm pipeing in json--but only when I'm piping out. Otherwise help goes to stderr and nothing goes to stdout.

I do honestly think jq is a cool and powerful tool. I also appreciate little things like auto-color when appropriate--git also does this. Git also uses your pager, which might trivialize my personal use case.

Re: Show HN: Catj – A new way to display JSON files

#117
post #5

Have you seen gron[0]? It's similar: flattens JSON documents to make them easily greppable. But it also can revert (ie, ungron) so you can pipe json to gron, grep -v to remove some data, then ungron to get it back to json. [0] https://github.com/tomnomnom/gron

What does grep + gron give you over jq?

Wrong question: It's not a competition.

There are cases when you have some complicated json and just want to search for stuff. Then you use grep + gron.

There are cases when you want a complete json processing tool. Then you use jq.

You can probably simulate each approach with the other approach, but the code needed to this is just too tedious to write. So you use whatever tool fits your use case.

Re: Show HN: Catj – A new way to display JSON files

#119
post #36

Earlier quoted context omitted.

Would probably be shorter if you used `tostream`

You're right. Good tip: jq -r ' tostream | select(length > 1) | ( .[0] | map( if type == "number" then "[" + tostring + "]" else "." + . end ) | join("") ) + " = " + (.[1] | @json) ' EDIT: For those who want to save this script, you can put just the jq code in an executable file with the shebang: #!/usr/bin/jq -rf

This would be more suitable to large json files if used with the `--stream` flag. Here's my take on it:

  jq -c --stream '
    . as $in 
    | select(length == 2) 
    | (
      $in[0] | map(
        if type == "number" 
        then "[" + tostring + "]" 
        else "." + . 
        end
      ) | add
    ) + " = " + ($in[1] | tostring)'
Using `--stream` allows jq to start before parsing the entire json file. In my experience, a 700mb json file can take up 5gb of ram in either jq or python -m json.

Re: Show HN: Catj – A new way to display JSON files

#120
post #117

Earlier quoted context omitted.

What does grep + gron give you over jq?

Wrong question: It's not a competition. There are cases when you have some complicated json and just want to search for stuff. Then you use grep + gron. There are cases when you want a complete json processing tool. Then you use jq. You can probably simulate each approach with the other approach, but the code needed to this is just too tedious to write. So you use whatever tool fits your use case.

No, but you still have to make a decision about which tool to use. So it's helpful to have a sense of the use cases for each.
Post reply on HN