Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jshon: A command line program to parse, read and create JSON

kmkeen.com

Post reply on HN
Print view View original

11–20 of 21 posts

Re: Jshon: A command line program to parse, read and create JSON

#11

Post by keenerd »

justincormack wrote: post #5

There have been some attempts to make a line oriented XML format so that shell tools work; doing the same thing with JSON is possible too; there are a few ways to do it. Have been meaning to give it a go at some point. In my view it makes more sense than building a new set of tools, especially as the Jshon ones do not really seem to compose properly. References for XML: http://www.lbreyer.com/unix_xml-1.html http://w…

> the Jshon ones do not really seem to compose properly.

Could you elaborate on that a bit?

Re: Jshon: A command line program to parse, read and create JSON

#12

Post by reirob »

justincormack wrote: post #5

There have been some attempts to make a line oriented XML format so that shell tools work; doing the same thing with JSON is possible too; there are a few ways to do it. Have been meaning to give it a go at some point. In my view it makes more sense than building a new set of tools, especially as the Jshon ones do not really seem to compose properly. References for XML: http://www.lbreyer.com/unix_xml-1.html http://w…

I have been using XML Starlet (http://xmlstar.sourceforge.net/) for many years in combination with Unix Shell. It is a very nice command line tool allowing to extract data from XML documents (using XPath expressions), to edit XML documents and even to convert XML to PYX format. The only problem it has so far is with big XML files as it loads them in the memory. But there is work going on to allow extract data without loading whole XML file in RAM.

Re: Jshon: A command line program to parse, read and create JSON

#15

Post by justincormack »

keenerd wrote: post #11
justincormack wrote: post #5

There have been some attempts to make a line oriented XML format so that shell tools work; doing the same thing with JSON is possible too; there are a few ways to do it. Have been meaning to give it a go at some point. In my view it makes more sense than building a new set of tools, especially as the Jshon ones do not really seem to compose properly. References for XML: http://www.lbreyer.com/unix_xml-1.html http://w…

> the Jshon ones do not really seem to compose properly. Could you elaborate on that a bit?

Most of the operations do not map any json to json, so you cant pipe them. The whole design principle of the unix command line tools is that they all map lines to lines so anything is composable.

So for json I would want a structure match corresponding to grep, filter and map operations, merges etc each of which maps valid json to valid json.

Re: Jshon: A command line program to parse, read and create JSON

#16

Post by keenerd »

justincormack wrote: post #15
keenerd wrote: post #11

Earlier quoted context omitted.

> the Jshon ones do not really seem to compose properly. Could you elaborate on that a bit?

Most of the operations do not map any json to json, so you cant pipe them. The whole design principle of the unix command line tools is that they all map lines to lines so anything is composable. So for json I would want a structure match corresponding to grep, filter and map operations, merges etc each of which maps valid json to valid json.

Almost half* of the commands map json (stdin) to json (stdout). Extract, modify and insert all take and spit out json. Originally (when it only supported single commands) this is how you had to do everything. But that much piping is a real drag. Anyway, show me a shell json util that is more composable or at least provide a suggestion to improving it :-)

Grep/filter/map are all stuff you do in the shell, after using Jshon to extract the data from the hard-to-parse-safely json.

*The remaining commands are concessions to make line-oriented shell processing easier.

Re: Jshon: A command line program to parse, read and create JSON

#17

Post by keenerd »

forgotusername wrote: post #2

Note for simple uses, Python 2.6+ comes with the "json.tool" module (also part of simplejson if you're $ type json json is aliased to `python -m json.tool' $ curl -s twitter.com/users/BarackObama.json|json|grep '"url"' "url": "http://www.barackobama.com",

Please never use grep on json. If you are going to use python as a heavyweight json_reformat, at least have the decency to use all of python's excellent json lib.

Re: Jshon: A command line program to parse, read and create JSON

#19

Post by keenerd »

keenerd wrote: post #8
VMG wrote: post #6

Great tool, was thinking about writing something like that myself. I especially wanted a tool that would print the value of a specific key to STDOUT with a syntax like echo "{ 'a': {'b': {'c": 0 } } }" | json-tool "a.b.c" # echos 0 Maybe your tool does that but I missed it

Glad you like it. I'll be adding a path-like syntax next. This was just a little one-day hackup to learn getopts. You can do similar with -e a -e b -e c. Or just make your own wrapper with jshon $(echo ".a.b.c" | sed 's/./ -e/g')

jshon $(sed 's/\./ -e/g' <<< ".a.b.c")

Re: Jshon: A command line program to parse, read and create JSON

#20

Post by webXL »

akv wrote: post #14

It is a bit long, but you can also use Python for parsing the json files from command line: echo '{"a": "hello", "b": "world"}' | python -c"import sys,json;j = json.loads(sys.stdin.read());print j['a']"

or in 2.6:

  echo '{"a": "hello", "b": "world"}' | python -mjson.tool
Post reply on HN