Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

JC – JSONifies the output of many CLI tools

kellyjonbrazil.github.io

Post reply on HN
Print view View original

91–100 of 136 posts

Re: JC – JSONifies the output of many CLI tools

#91

Post by artemisart »

djedr wrote: post #55
thesuperbigfrog wrote: post #50

Earlier quoted context omitted.

>> something formatted with pipes, quotes and spaces would be better How well would this format handle deeply nested structures? It seems like it would require a lot of space characters compared to nesting open and close characters: {} or () or [] How would escaping pipes, quotes, and spaces work to represent those character literals? There are already numerous structured text formats: JSON, XML, S-expressions, YAML,…

Dare I suggest Jevko[0] as yet another alternative? eth0 [ ip [127.15.34.23] flags [[BROADCAST][UNICAST]] mtu [1500] name ["Gigabit" by Network Interfaces Inc.] ] This is one of the things it was designed with in mind. It's even simpler and more flexible than S-expressions. Handles deeply nested structures perfectly well. Has only 3 characters to escape (brackets and the escape character). (I am the author) [0] https…

How do you differentiate types with jevko (numbers, strings, boolean)? Your examples on jevko.org appear lossy as they encode in the same way things that are different in JSON and I don't know how you would then differentiate between true and "true", 27 and "27", etc.

Re: JC – JSONifies the output of many CLI tools

#93

Post by hnlmorg »

brodo wrote: post #34
spinningslate wrote: post #10

Didn't know about this, the HN dividend pays out again! When wrestling with sed/awk in trying to parse results of a shell command, I've often thought that a shell-standard, structured outpout would be very handy. Powershell[0] has this, but it's a binary format - so not human-readable. I want something in the middle: human- and machine-readable. Without either having to do parsing gymnastics. jc isn't quite that shel…

Nushell has this too. I‘ve tried is as daily driver for a while. It‘s not there yet, but almost. After it hits 1.0 I‘m going to switch for good and leave the duck tape solutions behind.

Shameless plug but maybe give murex a look. That’s been stable for a while now and does the same thing too.

HTTPS://GitHub.com/lmorg/murex

Re: JC – JSONifies the output of many CLI tools

#94

Post by WorldMaker »

lstamour wrote: post #85
anonymoushn wrote: post #79

Earlier quoted context omitted.

I'm not sure what you are answering. I am writing about the thing where some programs intentionally write different byte streams to stderr and stdout that will be interleaved a certain way if stdout and stderr turn out the be the same thing, but if the user wishes to distinguish which bytes were written to which stream, they can no longer recover the correct interleaving of the bytes.

The best compromise I can think of would be to prepend the line source to the combined output, so lines in the combined output can be distinguished. But depending on how you do that, you might have timing issues. In fact, any form of interleaving the outputs of stderr and stdout are technically subject to timing issues because even the source application can’t specify that bytes in one stream should follow bytes in a…

You could require orderable sequence numbers before or after each line. It would be easiest as yet another VT escape sequence, but if you are pie-in-sky adding more standard "files" anyway maybe you wish for lines to be newline-delimited JSON CRDTs while you are at it.

Re: JC – JSONifies the output of many CLI tools

#95

Post by WorldMaker »

HyperSane wrote: post #92

The greatest thing about powershell is that all commands returns structured output like this.

Also, this tool is useful for using old unix commands in powershell: `jc … | ConvertFrom-Json` easily gives you powershell objects to work with.

Re: JC – JSONifies the output of many CLI tools

#96

Post by djedr »

artemisart wrote: post #91
djedr wrote: post #55

Earlier quoted context omitted.

Dare I suggest Jevko[0] as yet another alternative? eth0 [ ip [127.15.34.23] flags [[BROADCAST][UNICAST]] mtu [1500] name ["Gigabit" by Network Interfaces Inc.] ] This is one of the things it was designed with in mind. It's even simpler and more flexible than S-expressions. Handles deeply nested structures perfectly well. Has only 3 characters to escape (brackets and the escape character). (I am the author) [0] https…

How do you differentiate types with jevko (numbers, strings, boolean)? Your examples on jevko.org appear lossy as they encode in the same way things that are different in JSON and I don't know how you would then differentiate between true and "true", 27 and "27", etc.

A plain Jevko parser simply turns your unicode sequence into a tree which has its fragments as leaves/labels.

No data types on that level, much like in XML.

Now above that level there is several ways to differentiate between them.

The simplest pragmatic way is a kind of type inference: if a text parses as a number, it's a number, if it's "true" or "false", it's a boolean. Otherwise it's a string. If you know the implicit schema of your data then this will be sufficient to get the job done.

Otherwise you employ a separate schema -- JC in particular has per-parser schemas anyway, so that's covered in this case. If it wouldn't, you'd need to write a schema yourself.

Or you do "syntax-driven" data types, similar to JSON, e.g. strings start w/ "'".

Here is a shitty demo: https://jevko.github.io/interjevko.bundle.html

It shows schema inference from JSON and the schemaless (syntax-driven) flavor.

Jevko itself is stable and formally specified: https://github.com/jevko/specifications/blob/master/spec-sta...

It's very easy to write a parser in any language (I've written one in several) and from there start using it.

However, I am still very much working on specifications for formats above Jevko. I have some recent implementations of the simplest possible format which converts Jevko to arrays/objects/strings:

* https://github.com/jevko/easyjevko.lua

* https://github.com/jevko/easyjevko.js

The schema-driven format that was used in the demo is implemented here:

* https://github.com/jevko/interjevko.js

* https://github.com/jevko/jevkoschema.js

Re: JC – JSONifies the output of many CLI tools

#97

Post by codedokode »

psychoslave wrote: post #89
codedokode wrote: post #47

I think that JSON is a bad choice here. It is obvious that CLI commands should produce machine-readable output because they are often used in scripts, and accept machine-readable input as well. Using arbitrary text output was a mistake because it is difficult to parse, especially when spaces and non-ASCII characters are present. A good choice would be a format that is easily parsed by programs but still readable by t…

Following the project documentation, you easily come to: jc dig example.com | jq [ { "id": 30081, "opcode": "QUERY", "status": "NOERROR", "flags": [ "qr", "rd", "ra" ], "query_num": 1, "answer_num": 1, "authority_num": 0, "additional_num": 1, "opt_pseudosection": { "edns": { "version": 0, "flags": [], "udp": 4096 } }, "question": { "name": "example.com.", "class": "IN", "type": "A" }, "answer": [ { "name": "example.c…

For me there are too many quotes and brackets. My proposed format can also be converted to JSON if necessary.

Re: JC – JSONifies the output of many CLI tools

#98

Post by codedokode »

onion2k wrote: post #66
codedokode wrote: post #47

I think that JSON is a bad choice here. It is obvious that CLI commands should produce machine-readable output because they are often used in scripts, and accept machine-readable input as well. Using arbitrary text output was a mistake because it is difficult to parse, especially when spaces and non-ASCII characters are present. A good choice would be a format that is easily parsed by programs but still readable by t…

If you use something other than JSON you'd have to wait until every app you want to use chooses to update to support your preferred format. That might take a while. Wouldn't it be better to use JSON for the output as that's an acceptable input to lots and lots of applications already, and if you want to read the output just pass it to an app that converts from JSON to "something formatted with pipes, quotes and space…

On the other hand, if machine-readable format is adopted, it will be used for many years or even decades. So instead of making a quick hack that everybody will regret later it might be better to spend some time comparing different options.

Re: JC – JSONifies the output of many CLI tools

#99

Post by codedokode »

kbrazil wrote: post #74
codedokode wrote: post #47

I think that JSON is a bad choice here. It is obvious that CLI commands should produce machine-readable output because they are often used in scripts, and accept machine-readable input as well. Using arbitrary text output was a mistake because it is difficult to parse, especially when spaces and non-ASCII characters are present. A good choice would be a format that is easily parsed by programs but still readable by t…

Hi there - `jc` author here. `jc` can also output in YAML format with the `-y` flag. It is fairly trivial to add other options in the future since `jc` just turns the text into objects which can be serialized to many different formats. For example: % jc -y date --- year: 2022 month: Nov month_num: 11 day: 3 weekday: Thu weekday_num: 4 hour: 9 hour_24: 9 minute: 0 second: 22 period: AM timezone: PDT utc_offset: day_of…

Great, by I would prefer to avoid YAML because it is very complicated and difficult to parse.

Re: JC – JSONifies the output of many CLI tools

#100

Post by WorldMaker »

codedokode wrote: post #47

I think that JSON is a bad choice here. It is obvious that CLI commands should produce machine-readable output because they are often used in scripts, and accept machine-readable input as well. Using arbitrary text output was a mistake because it is difficult to parse, especially when spaces and non-ASCII characters are present. A good choice would be a format that is easily parsed by programs but still readable by t…

> A good choice would be a format that is easily parsed by programs but still readable by the user.

I think the powershell approach is a good one here too: powershell commands output binary streams of objects rather than text and it is powershell itself that has several standard ways of human readable outputs, most of which are automatic (but easily tweaked with an extra pipe or two). Standard human readable forms are nice, and even standardized there's no need to rely on parsing them back out into objects because they are already passed as objects so they can focus a bit more on "pretty" over "parse-able" (such as including human useful things like ellisions `…` on long columns).

Post reply on HN