Live data from Hacker News

Curl will now output JSON

daniel.haxx.se

21–29 of 29 posts

Re: Curl will now output JSON

#21

Earlier quoted context omitted.

libxo is doing a good job for FreeBSD. It would be nice if Linux also adapts that, or something similar, one day. Eg I have a script that needs output from netstat, and `netstat -bin --libxo json | jq ...` is a great way to get that info instead of scraping the human-readable output.

Sorry, stupid question, but why not use /proc/net directly?

/proc is a linuxism.

Re: Curl will now output JSON

#22
post #7

Earlier quoted context omitted.

It's nice but I don't think json is such a great format for it. The overhead of parsing it is incredibly annoying along with being unable to pack binary data which is what unix tools can deal with to some extent

i have 2 letters for you: jq any tool that outputs json + jq => instant scriptability win.

Particularly if you prepend queries with try. jq likes to throw errors when unexpected things happen in your search or your data, and it may return "null". Using try instead returns an empty string.

When walking JSON structures to validate data I code each validation separately instead of burying them in blocks of if-then-else logic. I think this makes it easier to maintain and more obvious when dealing with data that for example may be in a list or an array of lists.

But yeah if you write golang scripts and want to shuffle data in and out of bash, jq is your #1 power tool.

Re: Curl will now output JSON

#23
post #2

I think if all command line tools outputted JSON, it would go a long way towards fixing the "ad-hoc parser problem" in scripts, and bring unixy shells around to a more PowerShell-like object-passing style. Passing plain text around is a great idea, but It's silly to restrict yourself to a single structural unit within that text (the "line"). We have better tech now.

It's slow and obtuse, the syntax of processing commands is too verbose, there are surprises with null values and return codes jq gives.

It's not the answer for everything, it's just one of the tools. It feels cosy to huge amount of people because it's the product of web explosion, but at the core it's accidental and not that good.

Re: Curl will now output JSON

#24
post #7

Earlier quoted context omitted.

It's nice but I don't think json is such a great format for it. The overhead of parsing it is incredibly annoying along with being unable to pack binary data which is what unix tools can deal with to some extent

i have 2 letters for you: jq any tool that outputs json + jq => instant scriptability win.

Two letters, but then several lines to write an actual simple request (with some dances to battle the null and the exit code), and it's slow.

Re: Curl will now output JSON

#27
post #2

I think if all command line tools outputted JSON, it would go a long way towards fixing the "ad-hoc parser problem" in scripts, and bring unixy shells around to a more PowerShell-like object-passing style. Passing plain text around is a great idea, but It's silly to restrict yourself to a single structural unit within that text (the "line"). We have better tech now.

Maybe a good interim solution would be a “proxy command” that knows the output format of common commands, and how to convert them to json. So for example you could pipe the output of ls to this tool, and it would be smart enough to parse it and convert to json.

https://github.com/kellyjonbrazil/jc was posted in another comment; it appears to be exactly that.

Re: Curl will now output JSON

#28
post #2

I think if all command line tools outputted JSON, it would go a long way towards fixing the "ad-hoc parser problem" in scripts, and bring unixy shells around to a more PowerShell-like object-passing style. Passing plain text around is a great idea, but It's silly to restrict yourself to a single structural unit within that text (the "line"). We have better tech now.

libxo is doing a good job for FreeBSD. It would be nice if Linux also adapts that, or something similar, one day. Eg I have a script that needs output from netstat, and `netstat -bin --libxo json | jq ...` is a great way to get that info instead of scraping the human-readable output.

It would be great to have libxo in linux!

FYI, `jc netstat` also generates JSON output for `netstat`.

(disclosure: I'm the author of `jc`)

While I'm at it... check out `jtbl` on my github which prints the JSON or JSON Lines output to a table in your terminal.

https://github.com/kellyjonbrazil/jc

https://github.com/kellyjonbrazil/jtbl

Re: Curl will now output JSON

#29

Earlier quoted context omitted.

Sorry, stupid question, but why not use /proc/net directly?

/proc is a linuxism.

Yes... But even on Linux I've been bitten so much by changes in 'standard' tools just on successive versions of centos, the last 10 years... ifconfig changing its output syntax (I know 'just use ip' well it changes sometimes too...), hostname sets a transient hostname now, hostnames with _ are now refused... netstat had some subtle changes some time ago too. free changed too. df, fricking df output format changed. So now, I just hit /proc and /sys, and document the hell out of it. Sure, new fields can appear there but they're new columns. The parser has to handle 'additional' columns. I trust Linus more with not-breaking-userland than the folks that make 'portable' tools.

Glad I don't have to support other OSes...

Post reply on HN