Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bringing the Unix philosophy to the 21st century (2019)

blog.kellybrazil.com

Post reply on HN
Print view View original

61–70 of 151 posts

Re: Bringing the Unix philosophy to the 21st century (2019)

#61

Post by u801e »

JJMcJ wrote: post #54
c0l0 wrote: post #19

So I've been writing shell scripts for about two decades, about 75% of that time professionally. Parsing the unstructured, text-based output of utilities is not the problem for anyone who's had maybe a few weeks of training. Most `... | grep ... | cut ... | sed ... | awk ...`-abominations the post laments can be replaced by a single informed call to `awk`, making everything a lot more elegant and concise. Having JSON…

> awk Or Perl. That's what Perl was originally created for. Or sed. If awk is too mainstream for you. Or Python. Or just write the ugly pipeline in your script, and promptly forget how it actually works. I've done that a lot.

> Or sed

After learning how to use ed in scripts, I've found it's actually easier to use it compared to sed because it effectively has random access through the input rather than going from beginning to end.

But both awk and Perl have the advantage of storing values in variables over sed or ed.

Re: Bringing the Unix philosophy to the 21st century (2019)

#62

Post by 10000truths »

mcswell wrote: post #53

While you're at it, bring Unix into the 21st century, in which we use Unicode (UTF-8 encoding). The number of Unix utilities that badly support Unicode is, well, painful to those of us who deal with non-ASCII data all the time.

Which utilities don’t support UTF8? I’ve never had encoding issues when using coreutil binaries e.g. grep or sed or awk. I guess if you want to display emojis or something, that might not work properly, but that’s an issue with your terminal, not with the utility you’re running.

Re: Bringing the Unix philosophy to the 21st century (2019)

#63

Post by u801e »

pjmlp wrote: post #51
lfnoise wrote: post #42

Earlier quoted context omitted.

Yes. I never understood why JSON over s-exprs. The absence of maps is not a negative. S-exprs can represent maps. There are no maps in JSON, really anyway. It is just text. How that data is represented in memory is the output of parsing. You could just as well parse (dict (a 1)(b 2)(c 3)) into a hash table if you wanted. You could also have sets (set 1 2 3) or whatever other data structure.

Fear of parentheses, basically.

But json already has quotes, commas, brackets and braces :-)

Re: Bringing the Unix philosophy to the 21st century (2019)

#64

Post by laumars »

natt941 wrote: post #5

I can understand why the idea of more structured, object-like input and output is appealing, but after using PowerShell for a while, my take is that it's much harder to manipulate objects into a consistent format than it is to manipulate text. For instance, if you want to compare Azure DNS records with DNS records from a Windows server, it's a huge pain because Get-AzDnsRecordSet and Get-DnsServerResourceRecord retur…

There are better shells out there for handling structural data, like Murex and Elvish

Re: Bringing the Unix philosophy to the 21st century (2019)

#65

Post by [deleted] »

hackerbrother wrote: post #55
pdimitar wrote: post #29

Earlier quoted context omitted.

This is even more true for the ungodly long `jq` incantations that people write. It's like I get it, the old way is ugly and not always easy to decipher but at least it's shorter and your chances of understanding it are better. I've had both -- the classic piped chain of UNIX commands and various JSON-producing tools piped to `jq`. The former were still easier to work with.

(Sorry, this is my first time trying to do a formatted comment here.) What I like to do is comments like: /* * Collates * [ * { * id: 4 * dept: 'oncology', * name: 'Joe S.' * } * . * . * . * ] * * into * * { * 4: { // id * 'Joe. S': { // name * dept: oncology * } * . * . * . * } */ ( Then insert horrible one-liner that does the transformation. )

[deleted]

Re: Bringing the Unix philosophy to the 21st century (2019)

#66

Post by kaliszad »

c0l0 wrote: post #19

So I've been writing shell scripts for about two decades, about 75% of that time professionally. Parsing the unstructured, text-based output of utilities is not the problem for anyone who's had maybe a few weeks of training. Most `... | grep ... | cut ... | sed ... | awk ...`-abominations the post laments can be replaced by a single informed call to `awk`, making everything a lot more elegant and concise. Having JSON…

I can see the potential in Babashka and other Clojure-based systems from the approach point of view. There needs to be a simple integrated editor or much better readline implementation though that needs to work out of the box. That should improve the interactivity in a major way. We also need to have a set of very simple tools to work with files, network connections (e.g. traffic dumps, netcat) and other typical shell tasks. This isn't really as efficient when used interactively in the Clojure ecosystem or anywhere else from what I have seen besides just using e.g. ls, du, df, fdisk, tcpdump, nc ...

More long term, I have high hopes of having an environment above the kernel that basically does what /proc and /sys and others do currently. Programs equivalent to those in /bin /usr/bin etc. would be just dynamically loadable modules or built into the basic tool set. No shell scripts anymore, no random filesystems with custom formats for everything, just a JITed VM that has a strong set of tools but anybody can extend it with either stuff written in some Clojure-like language, or something compiling to the VM or something native that has some kind of interface (FFI?) to be usable from the programs running on the VM (e.g. for cryptographic stuff or stuff that needs to be as efficient as possible). We would also need something like SSH but for structured data that would support a SHELL/ REPL-like workflow as a byproduct but really be meant for more or less high performance, efficient communication (e.g. useable even for large file copy operations and such). In the end, parts of this system could connect to a in kernel VM (BPF?) and execute there but we would interact with them using the nice, structured REPL.

This would be a huge undertaking but I can't really see, how we can radically improve the efficiency of work with the current systems. It seems, we are mostly just patching old approaches to do new tricks and to me, it seems to be falling apart. The complexity we impose upon us is crushing and I don't think all of it is necessary.

Re: Bringing the Unix philosophy to the 21st century (2019)

#67

Post by pjmlp »

u801e wrote: post #63
pjmlp wrote: post #51

Earlier quoted context omitted.

Fear of parentheses, basically.

But json already has quotes, commas, brackets and braces :-)

Yeah, apparently finding misplaced quotes, commas, brackets and braces is magically easier than misplaced parentheses. :)

Re: Bringing the Unix philosophy to the 21st century (2019)

#68

Post by ajuc »

rurban wrote: post #50

Thankfully he didn't propose XML. Unfortunately it looks like he (and many others) really thinks that JSON is better, even if it's underspecified, thus leading to possible insecurities. See the recent jsonsec thread. (Undefined key ordering and duplicate handling) So I have to bring in jsmn.h to parse protocols? Sorry no. Been there, done that. We are pushing too much unnecessary JSON around already. Unix is also abo…

> even if it's underspecified, thus leading to possible insecurities

what it replaces (unstructured text) is much less secure so I don't think that counts against json

Re: Bringing the Unix philosophy to the 21st century (2019)

#69

Post by hnlmorg »

kbrazil wrote: post #31
pdimitar wrote: post #29

Earlier quoted context omitted.

This is even more true for the ungodly long `jq` incantations that people write. It's like I get it, the old way is ugly and not always easy to decipher but at least it's shorter and your chances of understanding it are better. I've had both -- the classic piped chain of UNIX commands and various JSON-producing tools piped to `jq`. The former were still easier to work with.

Yes, I have seen those too! That’s why I also wrote Jello, which is like jq but uses pure python without the boilerplate. Python is nearly universal now and typically easy to read, though more verbose. Jq is just as much a write-once tool as awk and perl for more complex queries. For simple attribute calls, though, it’s both terse and readable.

This is why I wrote murex shell (https://github.com/lmorg/murex), it's an alternative $SHELL, so you'd use it in place of Bash or Zsh, but it's optimised for modern DevOps tools. Which means JSON and YAML are first class citizens.

It's syntax isn't 100% POSIX compatible so there is some new stuff to learn but it works with all the existing POSIX tools and is more readable than AWK and Perl but while also being terse enough to write one liners.

Re: Bringing the Unix philosophy to the 21st century (2019)

#70

Post by phendrenad2 »

"Up until about 2013 it made just as much sense as anything to assume unstructured text was a good way to output data at the command line..."

But in 2013 a certain data format called JSON was standardized as ECMA-404..."

"Had JSON been around when I was born in the 1970’s Ken Thompson and Dennis Ritchie may very well have embraced it as a recommended output format to help programs “do one thing well” in a pipeline."

This whole post hinges on the theory that JSON is a revolutionary technology that no one had created something like before, and no one even considered creating before.

But that seems completely wrong, right?

"The Xerox Network Systems Courier technology in the early 1980s influenced the first widely adopted standard. Sun Microsystems published the External Data Representation (XDR) in 1987. XDR is an open format, and standardized as STD 67 (RFC 4506)"

https://en.wikipedia.org/wiki/Serialization

Post reply on HN