Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

JC – JSONifies the output of many CLI tools

kellyjonbrazil.github.io

Post reply on HN
Print view View original

51–60 of 136 posts

Re: JC – JSONifies the output of many CLI tools

#51

Post by bonzini »

codedokode wrote: post #47

I think that JSON is a bad choice here. It is obvious that CLI commands should produce machine-readable output because they are often used in scripts, and accept machine-readable input as well. Using arbitrary text output was a mistake because it is difficult to parse, especially when spaces and non-ASCII characters are present. A good choice would be a format that is easily parsed by programs but still readable by t…

> In my opinion, something formatted with pipes, quotes and spaces would be better:

Just pipe it into a JSON-to-YAML script like this:

    #! /usr/bin/python3
    from ruamel import yaml
    import json, sys, io
    print(yaml.dump(json.load(sys.stdin)))

Re: JC – JSONifies the output of many CLI tools

#52

Post by samuell »

ducktective wrote: post #4

Then pipe it into jq [1] to query parameters or build up a formatted string. Or pipe it into rq [2] to convert the format to yaml, toml etc. [1]: https://stedolan.github.io/jq/tutorial/ [2]: https://github.com/dflemstr/rq#format-support-status

I would wish for jq to be a really generic tool for working with structured data on the commandline, but I have a really hard time figuring out how to do e.g. conditional-based editing etc. Can't get my head around that, and don't find any info about it on the net. Seems even something that just supports SQL (upon JSON) would be better in this regard.

Re: JC – JSONifies the output of many CLI tools

#53

Post by bonzini »

vbezhenar wrote: post #49
thesuperbigfrog wrote: post #48

Earlier quoted context omitted.

>> Shell would benefit from Content-Type/Accept headers. Like you can specify that cat accepts text and jq accepts Json. Then `ip a` would output corresponding type automatically. That seems unnecessary. Traditionally, shells have always used text streams. JSON is just text that follows a given convention. Couldn't what you are describing be implemented by setting environment variables or using command line flags? Fo…

I don't follow. JSON is not really readable. I don't want to see JSON output ever except for script debugging. I want to see well formatted output. But at the same time I want to be able to write something like ip a | filter "[].address like 192.*" So when I'm typing `ip a` I expect to get output for human and when I'm piping it to `filter` program, I expect for those programs to exchange with JSON (and ideally `filt…

FWIW ip already has JSON support:

    ip -j a s | jq 'map(select((.addr_info | .[].local)|startswith("192.168."))) | map(.ifname)'

Re: JC – JSONifies the output of many CLI tools

#54

Post by feanaro »

codedokode wrote: post #47

I think that JSON is a bad choice here. It is obvious that CLI commands should produce machine-readable output because they are often used in scripts, and accept machine-readable input as well. Using arbitrary text output was a mistake because it is difficult to parse, especially when spaces and non-ASCII characters are present. A good choice would be a format that is easily parsed by programs but still readable by t…

Some alternative ideas for making JSON more readable:

- Pipe into gron (https://github.com/tomnomnom/gron) to get a `foo.bar.baz = val` kind of syntax.

- Pipe into visidata (https://www.visidata.org/) to get a spreadsheet-like editable view.

Re: JC – JSONifies the output of many CLI tools

#55

Post by djedr »

thesuperbigfrog wrote: post #50
codedokode wrote: post #47

I think that JSON is a bad choice here. It is obvious that CLI commands should produce machine-readable output because they are often used in scripts, and accept machine-readable input as well. Using arbitrary text output was a mistake because it is difficult to parse, especially when spaces and non-ASCII characters are present. A good choice would be a format that is easily parsed by programs but still readable by t…

>> something formatted with pipes, quotes and spaces would be better How well would this format handle deeply nested structures? It seems like it would require a lot of space characters compared to nesting open and close characters: {} or () or [] How would escaping pipes, quotes, and spaces work to represent those character literals? There are already numerous structured text formats: JSON, XML, S-expressions, YAML,…

Dare I suggest Jevko[0] as yet another alternative?

  eth0 [
    ip [127.15.34.23]
    flags [[BROADCAST][UNICAST]]
    mtu [1500]
    name ["Gigabit" by Network Interfaces Inc.]
  ]
This is one of the things it was designed with in mind.

It's even simpler and more flexible than S-expressions.

Handles deeply nested structures perfectly well. Has only 3 characters to escape (brackets and the escape character).

(I am the author)

[0] https://jevko.org/

Re: JC – JSONifies the output of many CLI tools

#56

Post by ape4 »

codedokode wrote: post #47

I think that JSON is a bad choice here. It is obvious that CLI commands should produce machine-readable output because they are often used in scripts, and accept machine-readable input as well. Using arbitrary text output was a mistake because it is difficult to parse, especially when spaces and non-ASCII characters are present. A good choice would be a format that is easily parsed by programs but still readable by t…

JSON is the lazy choice. I particularly dislike quoting keys (variable names). Relaxed JSON, for one, allows unquoted keys http://www.relaxedjson.org But that's just a small step - I am sure we (the community) could do better.

Re: JC – JSONifies the output of many CLI tools

#57

Post by thesuperbigfrog »

vbezhenar wrote: post #49
thesuperbigfrog wrote: post #48

Earlier quoted context omitted.

>> Shell would benefit from Content-Type/Accept headers. Like you can specify that cat accepts text and jq accepts Json. Then `ip a` would output corresponding type automatically. That seems unnecessary. Traditionally, shells have always used text streams. JSON is just text that follows a given convention. Couldn't what you are describing be implemented by setting environment variables or using command line flags? Fo…

I don't follow. JSON is not really readable. I don't want to see JSON output ever except for script debugging. I want to see well formatted output. But at the same time I want to be able to write something like ip a | filter "[].address like 192.*" So when I'm typing `ip a` I expect to get output for human and when I'm piping it to `filter` program, I expect for those programs to exchange with JSON (and ideally `filt…

I was saying that Accept Headers or "format negotiation" concepts that are typically used in client-server communications are a bit overkill for command line tools and shell pipelines.

I agree that human-readable text formats should be the default output formats for command line tools, but that easy-to-parse structured text output formats should be easy to specify with either environment variables or command line flags.

If I am writing a script and I am using tools that support a given structured output format and use environment variables or command line flags for output configuration it could work something like this:

    #!/bin/env script-interpreter
    export PREFERRED_OUTPUT_FORMAT="JSON"
    query-cli-tool | filter-cli-tool --output-format=json | combinator-cli-tool --input-format=json | pretty-formatter-tool > output_file
This would mean that the command line tools default to human-readable formats, but can still generate JSON or some other structured text format when configured to do so.

Re: JC – JSONifies the output of many CLI tools

#58

Post by ElectricalUnion »

synergy20 wrote: post #42

to make this great tool truly universal, it has to be written in c instead of python these days, then provide python|javascript|etc bindings if possible. I'd like to use it on embedded systems, where python is too large to fit. this tool can be widely deployed just like awk|sed|etc but it has to be in C for that.

But python is also universal? You can just bundle jc inside https://github.com/jart/cosmopolitan/tree/master/third_party... and it's as universal as it gets.

Re: JC – JSONifies the output of many CLI tools

#59

Post by mprovost »

vbezhenar wrote: post #29
spinningslate wrote: post #10

Didn't know about this, the HN dividend pays out again! When wrestling with sed/awk in trying to parse results of a shell command, I've often thought that a shell-standard, structured outpout would be very handy. Powershell[0] has this, but it's a binary format - so not human-readable. I want something in the middle: human- and machine-readable. Without either having to do parsing gymnastics. jc isn't quite that shel…

Shell would benefit from Content-Type/Accept headers. Like you can specify that cat accepts text and jq accepts Json. Then `ip a` would output corresponding type automatically.

Someone on this site suggested that programs open another filehandle along with stdout and stderr (stdjson) for their json output which struck me as a way to make this work in a backwards-compatible fashion.

Re: JC – JSONifies the output of many CLI tools

#60

Post by rjzzleep »

codedokode wrote: post #47

I think that JSON is a bad choice here. It is obvious that CLI commands should produce machine-readable output because they are often used in scripts, and accept machine-readable input as well. Using arbitrary text output was a mistake because it is difficult to parse, especially when spaces and non-ASCII characters are present. A good choice would be a format that is easily parsed by programs but still readable by t…

I also dislike jq, but this is a bit of a non issue IMHO. You could in theory add any kind of output transformer in theory. The codebase doesn't seem to be optimized for that yet, but it should be trivial to add.
Post reply on HN