Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bringing the Unix philosophy to the 21st century (2019)

blog.kellybrazil.com

Post reply on HN
Print view View original

21–30 of 151 posts

Re: Bringing the Unix philosophy to the 21st century (2019)

#21

Post by kbrazil »

c0l0 wrote: post #19

So I've been writing shell scripts for about two decades, about 75% of that time professionally. Parsing the unstructured, text-based output of utilities is not the problem for anyone who's had maybe a few weeks of training. Most `... | grep ... | cut ... | sed ... | awk ...`-abominations the post laments can be replaced by a single informed call to `awk`, making everything a lot more elegant and concise. Having JSON…

I don’t think the issue is that it’s hard to manually parse. The problem is that it’s hard for someone else to read your ad-hoc parser years later and reason about what you did if they need to modify it.

Disclaimer: I am the author of the article and JC.

Re: Bringing the Unix philosophy to the 21st century (2019)

#22

Post by jayd16 »

natt941 wrote: post #5

I can understand why the idea of more structured, object-like input and output is appealing, but after using PowerShell for a while, my take is that it's much harder to manipulate objects into a consistent format than it is to manipulate text. For instance, if you want to compare Azure DNS records with DNS records from a Windows server, it's a huge pain because Get-AzDnsRecordSet and Get-DnsServerResourceRecord retur…

You can always just thunk down to the text representation and do things that way. Having the structure is a strict plus, no?

Re: Bringing the Unix philosophy to the 21st century (2019)

#23

Post by eadmund »

Why not go all the way and use a format capable of expressing code and data? I refer, of course, to S-expressions.

They also have the benefit of properly handling numbers. Some might look at the absence of maps as a negative, but I think alists are preferable anyway due to their constant ordering.

Re: Bringing the Unix philosophy to the 21st century (2019)

#24

Post by kbrazil »

ape4 wrote: post #18

That `jc` command that converts non-json output to json is neat. But, um, seems slightly kludgy to be obvious. What if the `ifconfig` format changes slightly. I suppose its a stop-gap until all commands have a json output option.

If the ifconfig output changes slightly, then it will probably break the thousands of ad-hoc parser implementations buried in scripts worldwide. With this approach, only a central parser library (open source) needs to be maintained so it can be fixed quickly and robustly.

That being said, the goal is that command line tools that output useful data for scripts should have a structured output option like JSON while still keeping the human text output option so something like JC doesn’t even need to exist.

Re: Bringing the Unix philosophy to the 21st century (2019)

#25

Post by kbrazil »

eadmund wrote: post #23

Why not go all the way and use a format capable of expressing code and data? I refer, of course, to S-expressions. They also have the benefit of properly handling numbers. Some might look at the absence of maps as a negative, but I think alists are preferable anyway due to their constant ordering.

That’s a good debate to have. I settled on JSON due to its readability and ubiquity in web APIs. It’s something people all the way down and up the stack are very familiar with these days.

Re: Bringing the Unix philosophy to the 21st century (2019)

#26

Post by freshhawk »

Uh ... what about when I don't want to load the entire stream into memory before the next stage starts running? Are these implicit json arrays that stream out?

Or do we now have incompatible json shell tools and streaming text tools as a permanent fixture?

I'm excited about structured output ideas, but json? I'd much rather have streams of whitespace separated words than json. That's in that "No type system is better than a bad type system" metaphorical area.

I'll take grepping with a theoretically brittle regex over this jq[1] any day of the week.

[1] jq -nc --stream 'inputs | select(length==2) | select( [.[0][0,2,4]] == ["results", "data", "row"]) | [ .[0][6], .[1]] '

Re: Bringing the Unix philosophy to the 21st century (2019)

#27

Post by crdrost »

natt941 wrote: post #5

I can understand why the idea of more structured, object-like input and output is appealing, but after using PowerShell for a while, my take is that it's much harder to manipulate objects into a consistent format than it is to manipulate text. For instance, if you want to compare Azure DNS records with DNS records from a Windows server, it's a huge pain because Get-AzDnsRecordSet and Get-DnsServerResourceRecord retur…

It's that the languages suck.

I've tried writing a language around this but it's... rusty. You kind of want to pipe Get-AzDnsRecordSet and Get-DnsServerRrsourceRecord to the same interface structure and then build another transform to some other structure from the interface... The program wires together the transforms. So the programming paradigm becomes rather unconventional, you only write transformations of “events”, you address other shell commands by listening to outputs of their “eventspace,” etc. ... You get a sort of strange logic programming language where the pipe operator has to be slightly specialized every time because it always needs to transform a little as it pipes

Re: Bringing the Unix philosophy to the 21st century (2019)

#28

Post by bluecatswim »

richardanaya wrote: post #2

I was literally just thinking about this a few days ago. I'm super excited by https://www.nushell.sh/ . I think they are hitting on an order of magnitude improvement paradigm of shells that fit very nicely with the theme of this article.

I've always been looking for anushell.

Re: Bringing the Unix philosophy to the 21st century (2019)

#29

Post by pdimitar »

kbrazil wrote: post #21
c0l0 wrote: post #19

So I've been writing shell scripts for about two decades, about 75% of that time professionally. Parsing the unstructured, text-based output of utilities is not the problem for anyone who's had maybe a few weeks of training. Most `... | grep ... | cut ... | sed ... | awk ...`-abominations the post laments can be replaced by a single informed call to `awk`, making everything a lot more elegant and concise. Having JSON…

I don’t think the issue is that it’s hard to manually parse. The problem is that it’s hard for someone else to read your ad-hoc parser years later and reason about what you did if they need to modify it. Disclaimer: I am the author of the article and JC.

This is even more true for the ungodly long `jq` incantations that people write.

It's like I get it, the old way is ugly and not always easy to decipher but at least it's shorter and your chances of understanding it are better.

I've had both -- the classic piped chain of UNIX commands and various JSON-producing tools piped to `jq`. The former were still easier to work with.

Re: Bringing the Unix philosophy to the 21st century (2019)

#30

Post by jl6 »

c0l0 wrote: post #19

So I've been writing shell scripts for about two decades, about 75% of that time professionally. Parsing the unstructured, text-based output of utilities is not the problem for anyone who's had maybe a few weeks of training. Most `... | grep ... | cut ... | sed ... | awk ...`-abominations the post laments can be replaced by a single informed call to `awk`, making everything a lot more elegant and concise. Having JSON…

> what we'd really need is for all tool versions/variants output on all platforms (all GNU/Linux distros, all the BSDs, all embedded Linux variants, all commercial UNICES, etc.) to be the same, all the time

The industry has approximated this state of affairs by approximating a GNU+Linux monoculture. Doesn’t matter that BSD tar behaves differently if ~nobody uses it.

Post reply on HN