Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Catj – A new way to display JSON files

github.com

Post reply on HN
Print view View original

101–110 of 127 posts

Re: Show HN: Catj – A new way to display JSON files

#101

Post by jolmg »

krick wrote: post #56
avidal wrote: post #5

Have you seen gron[0]? It's similar: flattens JSON documents to make them easily greppable. But it also can revert (ie, ungron) so you can pipe json to gron, grep -v to remove some data, then ungron to get it back to json. [0] https://github.com/tomnomnom/gron

That's actually a nice lifehack. Much simplier than jq. Unfortunately, would be harder to make all kinds of logical conditions for which jq allows (even if not that intuitively). It still feels like there must be something in between, some way to make queries with json more naturally, than with jq, yet with enough power.

jq is certainly a unique language, which makes it unfamiliar to work with. Intuitiveness and natural feeling comes when one has gotten familiar with it after a bit of practice and reading the manual, though. It's a very well thought-out language. A very nice design.

It might help to recognize how it's influenced by shell languages and XPath, if you're familiar with those.

Re: Show HN: Catj – A new way to display JSON files

#102

Post by geofft »

hokus wrote: post #83

this is something between a joke and a thought experiment. function cason(x){ switch(x[0]){ case "movie": switch(x[1]) { case "name" : return "Interstellar"; case "year" : return 2014; case "is_released": return true; case "director" : return "Christopher Nolan"; case "cast": switch(x[2]){ case 0: return "Matthew McConaughey"; case 1: return "Anne Hathaway"; case 2: return "Jessica Chastain"; case 3: return "Bill Irw…

Well, that's kind of the inverse of writing switch statements like

    def license(kernel):
        return {"Linux": "GPL",
                "FreeBSD": "BSD",
                "NT": "Proprietary"}[kernel]

Re: Show HN: Catj – A new way to display JSON files

#103

Post by geofft »

bborud wrote: post #92

Oh no. This looks like the horrible config format a colleague of mine invented at Yahoo when making an absolutely horrific config system. This brings back bad memories. This may look cute but it is horrific when dealing with large configs and you have to reconstruct all the structure in your head. Also, when you have a format that nests using brackets, braces and parenthesis, you can get help from the editor. This fo…

I think the idea is not that this is for storage or editing, but just for querying - you should keep your data in JSON, but if you want to find where something is, do `catj foo.json | grep something` and it'll tell you all the paths in the document where you can find the string "something". The intent is not to open catj format in a text editor, or to use the output of catj for anything other than ad-hoc purposes.

Re: Show HN: Catj – A new way to display JSON files

#104

Post by geofft »

emmelaich wrote: post #62
nfoz wrote: post #9

That's nice, I like that. Related, if you want more of a csv-style, see JSONLines. aka "newline-delimited JSON" http://jsonlines.org/

I don't see what jsonlines has over yaml; in fact the jsonlines examples presented there are almost trivially converted to yaml. e.g. the first example is valid yaml if you add a '- ' to each line. And JSON is (almost) a perfect subset of yaml. I've been using csv lately. It's reputation is overstated. What I like is that it's far more compact than yaml or json and trivially pulled into sqlite for ad-hoc queries.

One thing I like about JSON Lines is robustness to bad data - if an individual line is corrupted, you can discard it / print a warning and move on, and the parser can start again at the next newline. This makes it useful for log messages / metrics, because if something crashes while emitting a log line, you can recover. If something crashes while emitting an item in a YAML list, you might corrupt the entire rest of the document.

Another is that it makes streaming processing a little easier. Once you have a line, you know you can attempt to process it, and you can shard processing on newlines without a full YAML processor. Tools that work on newlines or tools that can just split on lines can handle the first level of JSON Lines output.

Re: Show HN: Catj – A new way to display JSON files

#105

Post by sdegutis »

jolmg wrote: post #72
hyperpallium wrote: post #48

Earlier quoted context omitted.

Appending | to each line and a last line . is a jq program that reproduces the original json. jq -r ' ( tostream | select(length > 1) | ( .[0] | map( if type == "number" then "[" + tostring + "]" else "." + . end ) | join("") ) + " = " + (.[1] | @json) + " |" ), "." '

That's insane. Here I was thinking about how much more challenging it would be to parse and reconstruct the object from jq, and you got the idea to take advantage of the syntax similarity to parse it as jq code itself. Nice. And so, that means the inverse of the jq code I posted would simply be: ( jq "$(sed 's/$/ |/;$a.')" As in: catj example.json \ | ( jq "$(sed 's/$/ |/;$a.')" original.json

Everything you guys have posted in this thread looks like pure ninja magic to me.

Re: Show HN: Catj – A new way to display JSON files

#106

Post by krick »

jolmg wrote: post #101
krick wrote: post #56

Earlier quoted context omitted.

That's actually a nice lifehack. Much simplier than jq. Unfortunately, would be harder to make all kinds of logical conditions for which jq allows (even if not that intuitively). It still feels like there must be something in between, some way to make queries with json more naturally, than with jq, yet with enough power.

jq is certainly a unique language, which makes it unfamiliar to work with. Intuitiveness and natural feeling comes when one has gotten familiar with it after a bit of practice and reading the manual, though. It's a very well thought-out language. A very nice design. It might help to recognize how it's influenced by shell languages and XPath, if you're familiar with those.

Well, no arguing there, it is indeed. And I use it from time to time. However, it's not like I need a tool like that every day, and if I'm no using it for a week I usually need to "learn" it all over again.

Re: Show HN: Catj – A new way to display JSON files

#108

Post by cmonnow »

sdegutis wrote: post #105
jolmg wrote: post #72

Earlier quoted context omitted.

That's insane. Here I was thinking about how much more challenging it would be to parse and reconstruct the object from jq, and you got the idea to take advantage of the syntax similarity to parse it as jq code itself. Nice. And so, that means the inverse of the jq code I posted would simply be: ( jq "$(sed 's/$/ |/;$a.')" As in: catj example.json \ | ( jq "$(sed 's/$/ |/;$a.')" original.json

Everything you guys have posted in this thread looks like pure ninja magic to me.

looks like ancient Greek for us plebs

Re: Show HN: Catj – A new way to display JSON files

#109

Post by geezerjay »

jolmg wrote: post #46
etxm wrote: post #41

Earlier quoted context omitted.

Holy moly, Your jq skills are savage.

Thanks, but I haven't really done much in jq before. Doing the above script involved looking a lot through the manual and experimenting. This exercise was a learning experience for me. I was only able to do this because the manpage is well written.

And that's how your skills gradually become savage. Kudos.

Re: Show HN: Catj – A new way to display JSON files

#110

Post by tedivm »

ficklepickle wrote: post #34
tedivm wrote: post #21

For exploring large files I released a program called "JSONSmash" that runs a shell which lets you browse the data as if it was a filesystem. https://blog.tedivm.com/open-source/2017/05/introducing-json...

What an interesting idea! On my last contract, I had to deal with 100mb+ JSON responses from a barely-documented API. This would have come in handy when I was figuring it out. I've used JSONExplorer for this purpose, but it is web based and doesn't handle files this large. Extending the filesystem metaphor to JSON data and re-using the same commands strikes me as a great idea. Did another project inspire you, or did…

I came up with this idea on my own when dealing with the AWS Bulk API files. I did make a "Show HN" but it got a total of four upvotes.
Post reply on HN