Live data from Hacker News

The Beauty of Unix Pipelines

prithu.xyz

281–290 of 388 posts

Re: The Beauty of Unix Pipelines

#281
post #267

Earlier quoted context omitted.

> - Bash is awful (should be replaced with Python, and there are so many Python-based shells now that reify that opinion) You know that no one uses Bash these days right? Fish shell, Zsh are the much newer and friendlier shells these days. All my shells run on Fish these days. I have to tell you that my shell ergonomics has improved a lot after I started using fish shell. But to your point, _why_ is Bash awful?

> You know that no one uses Bash these days right? I strongly disagree although, just like you, I have no evidence to support my claim

> I strongly disagree although, just like you, I have no evidence to support my claim

Haha. Fair enough!

Re: The Beauty of Unix Pipelines

#282
post #22

Earlier quoted context omitted.

Every time I see that sort of stuff I end up | perl, still after 20 odd years. Sometimes i fine perl is not installed and I die a little.

In what unix system perl is not installed? It is in the default install on current versions of macos (where it comprises the majority of interpreted programs), openbsd, freebsd, ubuntu and fedora linux.

In FreeBSD, Perl was removed from base 17 years ago.

* https://unix.stackexchange.com/a/568069/5132

Re: The Beauty of Unix Pipelines

#283
post #274

Earlier quoted context omitted.

- This fails on badly formed PATH, or malicious PATH - When PATH contains entries that no longer exist / are not mounted, substitution with "sed" gives a wildcard that is not expanded, and eventually makes "file" report an error, which is not filtered out - If PATH contains entries that have spaces, the expansion is incorrect

More realistically, it also fails if you have directories on your path that are symlinks to other directories in the path. In that case their programs are doubly-counted. Anyways, if your PATH is malicious then you have worse problems than this silly script :)

Your error is in thinking that either of those cases is in any way malice.

Re: The Beauty of Unix Pipelines

#284

Earlier quoted context omitted.

Indeed, and the shell lives and breathes spaces. Arrays of arguments are created every time one types a command. Without spaces-as-magic we’d be typing things like this all the time: $ exec(‘ls’, ‘-l’, ‘A B C’) Maybe that’s unrealistic? I mean, if the shell was like that, it probably wouldn’t have exec semantics and would be more like this with direct function calls: $ ls(ls::LONG, ‘A B C’) Maybe we would drop the pa…

I don't think this is the problem people usually complain about. The much bigger problem is that spaces make text-only commands compose badly. $ ls -l `find ./ -name *abc*` Is going to work very nicely if file names have certain properties (no spaces, no special chars), and it's going to break badly if they don't. Quoting is also very simple in simple cases, but explodes in complexity when you add variables and other…

That's still just a shell problem since the shell is expanding `find` and placing the unquoted result into the command line. Some non-POSIX shells don't have this issue. For example with murex you could use either of these two syntaxes:

    ls -l ${find ./ name *abc*}
    # return the result as a single string

    ls -l @{find ./ name *abc*}
    # return the result as an array (like Bourn shell)
So in the first example, the command run would look something like:

    ls -l "foo abc bar"
whereas in the second it would behave more like Bourne shell:

    ls -l foo abc bar
As an aside, the example you provided also wouldn't work in Bourne shell because the asterisks would be expanded by the shell rather than `find`. So you'd need to quote that string:

    ls -l `find ./ -name "*abc*"`
This also isn't a problem in murex because you can specify whether to audo-expand globbing or not.

With POSIX shells you end up with quotation mark soup:

    ls -l "`find ./ -name '*abc*'`"
(and we're lucky in this example that we don't need to nest any of the same quotation marks. It often gets uglier than this!)

Murex also solves this problem by supporting open and close quote marks via S-Expression-style parentheses:

    ls -l (${find ./ -name (*abc*)})
Which is massively more convenient when you'd normally end up having to escape double quotes in POSIX shells.

---

Now I'm not trying to advocate murex as a Bash-killer, my point is that it's specifically the design of POSIX shells that cause these problems and not the way Unix pipelines work.

Re: The Beauty of Unix Pipelines

#285

Unix pipes are the a 1970's construct, the same way bell bottom pants are. It's a construct that doesn't take into account the problems and scale of today's computing. Unicode? Hope your pipes process it fine. Video buffers? High perf? Fuggetaboutit. Piping the output of ls to idk what? Nice, I'll put it on the fridge.

An informed criticism would be based upon the knowledge that Doug McIlroy came up with the idea in the 1960s. (-:

Re: The Beauty of Unix Pipelines

#288

Pipes are wonderful! In my opinion you can’t extol them by themselves. One has to bask in a fuller set of features that are so much greater than the sum of their parts, to feel the warmth of Unix: (1) everything is text (2) everything (ish) is a file (3) including pipes and fds (4) every piece of software is accessible as a file, invoked at the command line (5) ...with local arguments (6) ...and persistent globals in…

> Unix is seriously uncool with young people at the moment. I intend to turn that around and articles like this offer good material.

I offer you my well wishes on that.

Re: The Beauty of Unix Pipelines

#289
post #261

Earlier quoted context omitted.

Can you give me any clue as to what execve does? I looked at the man page but none the wiser. Sounds like magic from what I read there. I'm from a Windows backgroud and not used to pipes.

Well you have a Windows background, so you know BASIC, right? And you know that BASIC has a CHAIN statement, right? (-: Yes, I know. You've probably never touched either one. But the point is that this is simply chain loading . It's a concept not limited to Unix. execve() does it at the level of processes, where one program can chain to another one, both running in a single process. But it is most definitely not magi…

Thanks. I use pipes, occasionally ( I admit I prefer to code in a scripting language ) in Windows. It’s just execve that I was not familiar with.

Re: The Beauty of Unix Pipelines

#290
post #276

Earlier quoted context omitted.

> Pipes are wonderful! It's wonderful only if compared to worse things, pretending that PowerShell is not a thing, and that Python doesn't exist. UNIX pipes are a stringly-typed legacy that we've inherited from the 1970s. The technical constraints of its past have been internalised by its proponents, and lauded as benefits . To put it most succinctly, the "byte stream" nature of UNIX pipes means that any command that…

The reality is that physically, it actually works like this: (process1,serialize) | (parse,process2,serialize) | ... But as soon as you involve the network or persistent storage, you need to do all that anyway . And one of the beauties is that the tools are agnostic to where the data comes from, or goes.

So you're saying optional things should be forced on everything, because the rare optional case needs it anyway?

Look up what the Export-Csv, ConvertTo-Json, and Export-CliXml, Invoke-Command -ComputerName 'someserver', or Import-Sql commands do in PowerShell.

Actually, never mind, the clear and consistent naming convention already told you exactly what they do: they serialise the structured pipeline objects into a variety of formats of your choosing. They can do this with files, in-memory, to and from the network, or even involving databases.

This is the right way to do things. This is the UNIX philosophy. It's just that traditional shells typically used on UNIX do a bad job at implementing the philosophy.

Post reply on HN