Live data from Hacker News

Bringing the Unix philosophy to the 21st century (2019)

blog.kellybrazil.com

41–50 of 151 posts

Re: Bringing the Unix philosophy to the 21st century (2019)

#41
Parsing ad-hoc text formats always seem very fragile, as single spacr character (or even newline character) could easily break the pipeline. Using a proper data structure could make scripts works more "properly" when encountering these edge cases.

Which format to use a as representation of data structure might be debatable, but I think JSON is a reasonable choice. (Formatted) JSON is readable by human and could be easily parsed by programs.

Re: Bringing the Unix philosophy to the 21st century (2019)

#42
post #23

Why not go all the way and use a format capable of expressing code and data? I refer, of course, to S-expressions. They also have the benefit of properly handling numbers. Some might look at the absence of maps as a negative, but I think alists are preferable anyway due to their constant ordering.

Yes. I never understood why JSON over s-exprs. The absence of maps is not a negative. S-exprs can represent maps. There are no maps in JSON, really anyway. It is just text. How that data is represented in memory is the output of parsing. You could just as well parse (dict (a 1)(b 2)(c 3)) into a hash table if you wanted. You could also have sets (set 1 2 3) or whatever other data structure.

Re: Bringing the Unix philosophy to the 21st century (2019)

#43
post #39
post #18

That `jc` command that converts non-json output to json is neat. But, um, seems slightly kludgy to be obvious. What if the `ifconfig` format changes slightly. I suppose its a stop-gap until all commands have a json output option.

True. When I try it with `date` on Debian Sid I get parser error: $ jc -p date jc: Error - date parser could not parse the input data. Did you use the correct parser? For details use the -d or -dd option.

This `date | jc --date` seems to be the way

Re: Bringing the Unix philosophy to the 21st century (2019)

#44
post #24
post #18

That `jc` command that converts non-json output to json is neat. But, um, seems slightly kludgy to be obvious. What if the `ifconfig` format changes slightly. I suppose its a stop-gap until all commands have a json output option.

If the ifconfig output changes slightly, then it will probably break the thousands of ad-hoc parser implementations buried in scripts worldwide. With this approach, only a central parser library (open source) needs to be maintained so it can be fixed quickly and robustly. That being said, the goal is that command line tools that output useful data for scripts should have a structured output option like JSON while sti…

You have a point but when when I write a script to parse a command's output I don't read/parse every line. I'd probably grep for the one line I want.

Re: Bringing the Unix philosophy to the 21st century (2019)

#45
post #22
post #5

I can understand why the idea of more structured, object-like input and output is appealing, but after using PowerShell for a while, my take is that it's much harder to manipulate objects into a consistent format than it is to manipulate text. For instance, if you want to compare Azure DNS records with DNS records from a Windows server, it's a huge pain because Get-AzDnsRecordSet and Get-DnsServerResourceRecord retur…

You can always just thunk down to the text representation and do things that way. Having the structure is a strict plus, no?

This is a much more complex text representation.

Mandating tab separated columns with a consistent quoting for embedded spaces would be a net benefit. And it would match today's tools well.

Re: Bringing the Unix philosophy to the 21st century (2019)

#46
post #19

So I've been writing shell scripts for about two decades, about 75% of that time professionally. Parsing the unstructured, text-based output of utilities is not the problem for anyone who's had maybe a few weeks of training. Most `... | grep ... | cut ... | sed ... | awk ...`-abominations the post laments can be replaced by a single informed call to `awk`, making everything a lot more elegant and concise. Having JSON…

About this few weeks training you mentioned for Unix-style parsing of unstructured text - what resources do you recommend?

Re: Bringing the Unix philosophy to the 21st century (2019)

#47
post #19

So I've been writing shell scripts for about two decades, about 75% of that time professionally. Parsing the unstructured, text-based output of utilities is not the problem for anyone who's had maybe a few weeks of training. Most `... | grep ... | cut ... | sed ... | awk ...`-abominations the post laments can be replaced by a single informed call to `awk`, making everything a lot more elegant and concise. Having JSON…

I know enough awk to know it solves my problem, but use it to little to remember how, and the development experience is never very clean.

What i would like is an cli awk ide. (Rolls of the tongue right? )

Ideally i could write ... | awk-ide "scripts/thing.awk" | ...

If it exists, run it. Otherwise have it open an editor session that runs an awk program as i type it on some buffered data, shows the result, and have some hints on what syntax/variables are available.

Re: Bringing the Unix philosophy to the 21st century (2019)

#48
post #23

Why not go all the way and use a format capable of expressing code and data? I refer, of course, to S-expressions. They also have the benefit of properly handling numbers. Some might look at the absence of maps as a negative, but I think alists are preferable anyway due to their constant ordering.

Mostly because S-expressions are terribly hard for humans to read... and everyone is already familiar with javascript syntax.

Re: Bringing the Unix philosophy to the 21st century (2019)

#49
post #23

Why not go all the way and use a format capable of expressing code and data? I refer, of course, to S-expressions. They also have the benefit of properly handling numbers. Some might look at the absence of maps as a negative, but I think alists are preferable anyway due to their constant ordering.

Specially because it keeps being forgotten that Lisp Machines and Interlisp-D workstations shells were basically a graphical based REPL.

To put it in 2021 terms, Jupiter Netbooks in 1980's instead of PDP-11 green phosphor terminals.

Re: Bringing the Unix philosophy to the 21st century (2019)

#50
Thankfully he didn't propose XML. Unfortunately it looks like he (and many others) really thinks that JSON is better, even if it's underspecified, thus leading to possible insecurities. See the recent jsonsec thread. (Undefined key ordering and duplicate handling)

So I have to bring in jsmn.h to parse protocols? Sorry no. Been there, done that. We are pushing too much unnecessary JSON around already.

Unix is also about KISS. Structured data are lines and paragraphs.

Post reply on HN