Curl will now output JSON • Hacker News Board

Re: Curl will now output JSON

#11

Post by quintonish » Tue, Mar 17, 2020, 11:38 PM UTC

Note that this is not titled, "Curl will now format JSON". This will "Make curl display information on stdout after a completed transfer"[1].

[1] https://curl.haxx.se/docs/manpage.html#-w

Re: Curl will now output JSON

#12

Post by theMiddleBlue » Tue, Mar 17, 2020, 11:45 PM UTC

I'm testing it, maybe I'm doing it wrong but I don't understand well if it's possible to show only the json output (and not both json and response body). I'm doing it with a trick:

curl -I -XGET -w '%{json}' https://www.google.com | tail -1 | jq

Re: Curl will now output JSON

#13

Post by enriquto » Tue, Mar 17, 2020, 11:52 PM UTC

No need to panic! You can still pipe it to gron [0] and recover useable output.

[0] https://github.com/TomNomNom/gron

Re: Curl will now output JSON

#14

Post by lillecarl » Wed, Mar 18, 2020, 12:41 AM UTC

nyberg wrote: post #7

dTal wrote: post #2

I think if all command line tools outputted JSON, it would go a long way towards fixing the "ad-hoc parser problem" in scripts, and bring unixy shells around to a more PowerShell-like object-passing style. Passing plain text around is a great idea, but It's silly to restrict yourself to a single structural unit within that text (the "line"). We have better tech now.

It's nice but I don't think json is such a great format for it. The overhead of parsing it is incredibly annoying along with being unable to pack binary data which is what unix tools can deal with to some extent

json is fast, according to https://github.com/xi-editor/xi-editor/blob/master/README.md... "I considered binary formats, but the actual improvement in performance would be completely in the noise. Using JSON considerably lowers friction for developing plug-ins, as it’s available out of the box for most modern languages, and there are plenty of the libraries available for the other ones."

It doesn't have to be a one size fits all solution, there are edge cases where json wouldn't be the right choice, but for many and most things i could think of that i use daily I'd be happy with json. If it's small binary data i guess it could be base64 encoded.

Querying network interfaces, directories, process information, systemd daemons or anything else, i see these things could do great if they just outputted json and my shell could treat it as an object. If a script is performance critical, maybe it's time to rewrite it in another language anyways?

Re: Curl will now output JSON

#15

Post by touisteur » Wed, Mar 18, 2020, 1:17 AM UTC

Arnavion wrote: post #10

dTal wrote: post #2

I think if all command line tools outputted JSON, it would go a long way towards fixing the "ad-hoc parser problem" in scripts, and bring unixy shells around to a more PowerShell-like object-passing style. Passing plain text around is a great idea, but It's silly to restrict yourself to a single structural unit within that text (the "line"). We have better tech now.

libxo is doing a good job for FreeBSD. It would be nice if Linux also adapts that, or something similar, one day. Eg I have a script that needs output from netstat, and `netstat -bin --libxo json | jq ...` is a great way to get that info instead of scraping the human-readable output.

Sorry, stupid question, but why not use /proc/net directly?

Re: Curl will now output JSON

#16

Post by monkey26 » Wed, Mar 18, 2020, 3:36 AM UTC

To bad this requires nightly.

Re: Curl will now output JSON

#17

Post by otterlicious » Wed, Mar 18, 2020, 4:40 AM UTC

theMiddleBlue wrote: post #12

I'm testing it, maybe I'm doing it wrong but I don't understand well if it's possible to show only the json output (and not both json and response body). I'm doing it with a trick: curl -I -XGET -w '%{json}' https://www.google.com | tail -1 | jq

    -o /dev/null

Re: Curl will now output JSON

#18

Post by chatmasta » Wed, Mar 18, 2020, 5:01 AM UTC

dTal wrote: post #2

I think if all command line tools outputted JSON, it would go a long way towards fixing the "ad-hoc parser problem" in scripts, and bring unixy shells around to a more PowerShell-like object-passing style. Passing plain text around is a great idea, but It's silly to restrict yourself to a single structural unit within that text (the "line"). We have better tech now.

Maybe a good interim solution would be a “proxy command” that knows the output format of common commands, and how to convert them to json. So for example you could pipe the output of ls to this tool, and it would be smart enough to parse it and convert to json.

Re: Curl will now output JSON

#19

Post by rantwasp » Wed, Mar 18, 2020, 5:13 AM UTC

nyberg wrote: post #7

dTal wrote: post #2

I think if all command line tools outputted JSON, it would go a long way towards fixing the "ad-hoc parser problem" in scripts, and bring unixy shells around to a more PowerShell-like object-passing style. Passing plain text around is a great idea, but It's silly to restrict yourself to a single structural unit within that text (the "line"). We have better tech now.

It's nice but I don't think json is such a great format for it. The overhead of parsing it is incredibly annoying along with being unable to pack binary data which is what unix tools can deal with to some extent

i have 2 letters for you: jq

any tool that outputs json + jq => instant scriptability win.

Re: Curl will now output JSON

#20

Post by phnofive » Wed, Mar 18, 2020, 5:20 AM UTC

ayyess wrote: post #5

dTal wrote: post #2

I think if all command line tools outputted JSON, it would go a long way towards fixing the "ad-hoc parser problem" in scripts, and bring unixy shells around to a more PowerShell-like object-passing style. Passing plain text around is a great idea, but It's silly to restrict yourself to a single structural unit within that text (the "line"). We have better tech now.

Have you seen jc[0] as an effort to wrap common utilities to output json? [0]: https://github.com/kellyjonbrazil/jc

I hadn’t; cheers!