Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

JC – JSONifies the output of many CLI tools

kellyjonbrazil.github.io

Post reply on HN
Print view View original

131–136 of 136 posts

Re: JC – JSONifies the output of many CLI tools

#131

Post by fabianthomas »

I'm really concerned about performance/latency with the json approach. I couldn't find anyone discussing that yet completely.

Even when using jq (written in C) my quick tests show that parsing json is really slow compared to parsing with simple unix tools like awk. I suspect that to come from the fact that the parser has to check the full json output first in order to print a result, while awk does not care about syntax of the output.

I compared two shell scripts both printing the ifindex of some network device (that is the integer in the first column) 10 times.

Using awk and head combined gives me 0,068s total time measured with the time command.

Using ip with the -j flag together with jq gives 0,411s.

Therefore the awk approach is 6 times faster. And here I used a binary (ip) that already supports json output and doesn't even need the mentioned jc.

While this whole test setup is somewhat arbitrary I experienced similar results in the past when writing shell scripts for, e.g., my panel. Reach out to me if you are interested in my test setup.

Re: JC – JSONifies the output of many CLI tools

#132

Post by Too »

While I can appreciate the immediate value this tool brings for some applications.

At the same time it’s an epitome of everything that is wrong with current software landscape. Instead of fixing the deficiencies in upstream, once and for all, we just keep piling more and more layers on top.

Programs having structural data and APIs inside, that get translated into human representation - only to be re-parsed again into structural form. What could possibly go wrong?

If you are already in any programming environment, many of the tools already have better built in APIs. I mean who needs an “ls” or timestamp parser. Just use os.listdir or equivalent. As someone previously pointed out in this thread, the ls parser is in fact already broken, unsurprisingly. Mixing tools made for interactive use in automation is never a good idea.

The Unix philosophy sounds romantic in theory, but need structural data, throughout, to work reliably in practice. Kids, go with the underlying apis unless your tool has structured output.

Re: JC – JSONifies the output of many CLI tools

#133

Post by Too »

artemisart wrote: post #44
berkes wrote: post #40

A slightly related pet-peeve: I don't like it when "random" commands "squat" the two-letter domain, or worse, the one-letter domain: t, jq, jc etc. In my perfect world (which, obviously doesn't exist), commands from tools "in the wild" are at least three letters long. With historical exceptions for gnutools: preferably they'd take the three-letter space, but two-letters (cd, ls, rm etc) is fine. Two letter space outs…

I don't believe it's really an issue in practice, your `jc` alias will just take the priority and you can easily add one for jsonquery -> `path/to/bin/jc`. I think good short command names can help adoption (like for ripgrep, fd) but it's true that we should have a race to squat all the 2 letter names.

Given this tool is likely to be used in a bunch of bash scripts, collisions with aliases can be a big problem.

Re: JC – JSONifies the output of many CLI tools

#134

Post by garfieldnate »

One of the features of Julia that I really like and wish would be adopted elsewhere is that the stringification methods take a context object that can indicate the string is to be presented in: plaintext, ansii-colored terminal, json, HTML, LaTeX, GraphViz, etc. It would have been amazing to have this kind of flexibility considered in the Unix tools.

That said, I would argue that JSONLines is a better universal output format when you're dealing with pipelines. If the output is one giant JSON array, then you have to wait for a long-running program to finish completely before the output can be passed on to the next long-running program. If you output one JSON line at a time as you process your input, then the next program in the pipeline can get started on processing already without waiting for the first to finish completely.

Re: JC – JSONifies the output of many CLI tools

#135

Post by khiqxj »

jwilk wrote: post #125
khiqxj wrote: post #69

There are at least 3 ways this can create bugs: - It has to parse output of commands which may or may not be intended to be parsed and may or may not have a predictable format. The only way to overcome this is if this program becomes one of the Big Four "UN*X command output -> data" converters - It casts things to "float/int" - Depending on who made this library, the output itself may not be strict / predictable. Per…

> Big Four "UN*X command output -> data" converters What do you mean?

I mean that there would be a big 4 such converters if people actually invested time into these "front ends" to CLIs.

Re: JC – JSONifies the output of many CLI tools

#136

Post by kbrazil »

garfieldnate wrote: post #134

One of the features of Julia that I really like and wish would be adopted elsewhere is that the stringification methods take a context object that can indicate the string is to be presented in: plaintext, ansii-colored terminal, json, HTML, LaTeX, GraphViz, etc. It would have been amazing to have this kind of flexibility considered in the Unix tools. That said, I would argue that JSONLines is a better universal outpu…

Good news - `jc` supports JSONLines output for many commands and file-types, too!
Post reply on HN