If I had to wish only one thing in context of shell scripts, it would be to be able to separate passing back the actual result from a function from stdout/stderr/logging without messing with boilerplate around non-standard file descriptors/named pipes/temp files etc.
Bringing the Unix philosophy to the 21st century (2019)
111–120 of 151 posts
Re: Bringing the Unix philosophy to the 21st century (2019)
#112Earlier quoted context omitted.
I don’t think the issue is that it’s hard to manually parse. The problem is that it’s hard for someone else to read your ad-hoc parser years later and reason about what you did if they need to modify it. Disclaimer: I am the author of the article and JC.
This is even more true for the ungodly long `jq` incantations that people write. It's like I get it, the old way is ugly and not always easy to decipher but at least it's shorter and your chances of understanding it are better. I've had both -- the classic piped chain of UNIX commands and various JSON-producing tools piped to `jq`. The former were still easier to work with.
Opinion: Today's shells should have structured data; `jq` is a symptom of a shell that can't handle structured data.
https://ilya-sher.org/2018/09/10/jq-is-a-symptom/
My bias: I'm the author of Next Generation Shell.
Re: Bringing the Unix philosophy to the 21st century (2019)
#113This is how you "integrate" `jc` with Next Generation Shell ( https://github.com/ngs-lang/ngs ):
data = ``jc PROGRAM ARGS ...``
`data` is now structured data that comes from the parsed JSON.
Example (run from your shell):
ngs -pl '``jc ifconfig``.filter({"name": /docker/}).ipv4_addr'
Re: Bringing the Unix philosophy to the 21st century (2019)
#114If I had to wish only one thing in context of shell scripts, it would be to be able to separate passing back the actual result from a function from stdout/stderr/logging without messing with boilerplate around non-standard file descriptors/named pipes/temp files etc.
What, besides the actual result, do you write to stout? (And shouldn't it go to stderr instead?)
Re: Bringing the Unix philosophy to the 21st century (2019)
#115Why not go all the way and use a format capable of expressing code and data? I refer, of course, to S-expressions. They also have the benefit of properly handling numbers. Some might look at the absence of maps as a negative, but I think alists are preferable anyway due to their constant ordering.
Mostly because S-expressions are terribly hard for humans to read... and everyone is already familiar with javascript syntax.
I feel that depends on the formatting. Non-pretty JSON isn't very readable either.
```
(dict
:name apple
:ip 192.168.1.1
:nested (dict
:property value
)
)
```
Is almost JSON.
Re: Bringing the Unix philosophy to the 21st century (2019)
#116So I've been writing shell scripts for about two decades, about 75% of that time professionally. Parsing the unstructured, text-based output of utilities is not the problem for anyone who's had maybe a few weeks of training. Most `... | grep ... | cut ... | sed ... | awk ...`-abominations the post laments can be replaced by a single informed call to `awk`, making everything a lot more elegant and concise. Having JSON…
PowerShell is just so much nicer to use than anything where text munging is the only way to do things, and it’s just as “pluggable” as Unix shell commands. And it can output text (or JSON or XML or YAML) or whatever you want easily by piping into relevant commands (or just not piping anywhere if you wanted text). I don’t imagine those are going anywhere anytime soon but I think it is wrong to say the system cannot be…
I would disagree and say that while PowerShell is well...powerful, it's not as pluggable as the Unix shell. Purely due to objects vs plaintext. PowerShell works off of .NET objects thus if I want to take output from one command and send it to another I have to verify that the receiving command can in fact receive the object type I'm sending. This is where the Unix shell to me is more 'pluggable' but it makes it more muddy as well. I can pipe plain-text information to any utility but to ensure that it will do what I want and not error out I may have to slice and dice my initial output a bit.
Maybe to you pluggable means an output object from a utility being the same input object as another utility? In this case, Unix isn't pluggable at all.
Re: Bringing the Unix philosophy to the 21st century (2019)
#117If I had to wish only one thing in context of shell scripts, it would be to be able to separate passing back the actual result from a function from stdout/stderr/logging without messing with boilerplate around non-standard file descriptors/named pipes/temp files etc.
What, besides the actual result, do you write to stout? (And shouldn't it go to stderr instead?)
Typically informational messages/log message. Ideally yes logging should go to stderr but some applications that intercept the file descriptors consider anything to stderr as error and it is not always possible to change the handling for those applications. E.g. Octopus Deploy[0] does this: it doesn't outright fail the step assuming other "successful" steps happen after that but if the last command writes to stderr, it would; I use explicit exit statements as a workaround to handle this.
[0]: https://octopus.com/
Re: Bringing the Unix philosophy to the 21st century (2019)
#118"Up until about 2013 it made just as much sense as anything to assume unstructured text was a good way to output data at the command line..." But in 2013 a certain data format called JSON was standardized as ECMA-404..." "Had JSON been around when I was born in the 1970’s Ken Thompson and Dennis Ritchie may very well have embraced it as a recommended output format to help programs “do one thing well” in a pipeline."…
I don’t follow. Your examples are of data formats standardized in the 80’s while Unix was developed in the 60’s and 70’s. JSON even existed before 2013, but the fact that it became a standard in addition to being popular is the point I was making.
The first version of UNIX may have been created in 1969, but it continued to evolve for 20ish years, and I don't think 1969 UNIX really resembled what we had in 1989. But then the UNIX world stagnated, because Linux people were obsessed with cloning System V. Sometime during the Linux era, text became "cheap" enough to be used as a data format. So perhaps I'm wrong, and JSON's time has come in the UNIX world after all.
Re: Bringing the Unix philosophy to the 21st century (2019)
#119Earlier quoted context omitted.
True. When I try it with `date` on Debian Sid I get parser error: $ jc -p date jc: Error - date parser could not parse the input data. Did you use the correct parser? For details use the -d or -dd option.
Odd. What locale are you using? Should work fine with C or en_UTF8, per the readme.
$ echo $LANG
en_DK.UTF-8
$ date
2021-08-23T08:07:02 CESTRe: Bringing the Unix philosophy to the 21st century (2019)
#120Earlier quoted context omitted.
I had to write a powershell script and my impression was that objects are a much worse interface than the worst text manipulation tricks. Because you are passing objects around it quickly becomes really hard to understand what is going on on the code and which object interfaces the script is using, because the object lives behind the pipe so to speak. It was my impression that it is too easy to create unreadable powe…
I'm not sure I understand exactly what you mean. I would rather read Select { $_.Property } than some awk or sed stuff to fetch the same thing any day. I'm not sure how this is different than most other modern languages used to write scripts.
While you can produce unreadable sphagetti in any language, powershell is just a poor mans Python without any real language features. I'd much rather use csharp(script) if I have a need to interface with some ms stuff (.net system libraries or whatever), powershell is just underwhelming and bloated without any real performance or productivity gains.
Unix shells, for all their idiosynchronacies, tend to focus on small bits of functionality directly composed from common shell or system utilities. The complexities lie in outdated design, not in the language or form.
We could definitely do with an upgrade to the underlying tooling, I don't find anything otherwise lacking in shell scripting otherwise. I don't agree with the OP that we need a web readable data format - perhaps the jq syntax could be useful but the best part of shell scripting is it is approachable by humans - if it is not you are doing too much and should be writing your missile control program in python or c, or similar. I dont think it should be easier to produce functioning junk nor should it be a requirement to master a data syntax in order to print a formated date. Maybe you need a different date program if you need to do something special.