Live data from Hacker News

The Beauty of Unix Pipelines

prithu.xyz

351–360 of 388 posts

Re: The Beauty of Unix Pipelines

#351

Earlier quoted context omitted.

> (1) everything is text And lists are space-separated. Unless you want them to be newline-separated, or NUL-separated, which is controlled by an option that may or may not be present for the command you're invoking, and is spelled completely differently for each program. Or maybe you just quote spaces somehow, and good luck figuring out who is responsible for inserting quotes and who is responsible for removing them…

I think this was a failure on behalf of early terminal emulator developers. End-of-field and end-of-record should have been supported by the tty (either as visual symbols or via some sort of physical distancing, etc even if portrayed just as a space and a new line respectively) so that the semantic definition could have been preserved/distinguished.

TTYs in general are some of the most outdated and archiac pieces of technology still used today. They need a serious overhaul, but the problem is, any major change will break pretty much all programs.

Re: The Beauty of Unix Pipelines

#352

Earlier quoted context omitted.

I agree. Personally I think the Linux kernel should have a compilation-time option to disallow various special characters in filenames such as ASCII control characters and colons, so users who want to maintain a sane system can give themselves the option to do so.

So um, how would you work with files that came from a foreign file system? Would the kernel just crash when it sees them? Would they be effectively untouchable, like filenames with a '/' encoded in them?

Previously, I proposed addressing this with a mount option and a filesystem volume flag [1]. The mount option would prevent creating new files/directories with "bad names" but would still allow accessing ones which already exist. The filesystem volume flag would make bad names on that volume be considered fsck errors. (If the kernel found such a bad name on a volume with the flag enabled, it should log that as a filesystem error, and either return an IO error, or just pretend the file doesn't exist – whatever Linux does now when it finds a file on disk whose name contains a null byte or forward slash.)

Using a mount option and/or filesystem volume flag means it works fine with legacy volumes which might contain those bad names. On such legacy volumes, either that volume flag is not supported by the filesystem, or else it is but is disabled for that particular volume. If you mount it with the mount option, you can still access bad names on that volume, you just can't create new ones; if you need to create new bad names, you just (re)mount it with the mount option disabled.

[1] https://news.ycombinator.com/item?id=22873153

Re: The Beauty of Unix Pipelines

#353
post #307

Earlier quoted context omitted.

I don't think this is the problem people usually complain about. The much bigger problem is that spaces make text-only commands compose badly. $ ls -l `find ./ -name *abc*` Is going to work very nicely if file names have certain properties (no spaces, no special chars), and it's going to break badly if they don't. Quoting is also very simple in simple cases, but explodes in complexity when you add variables and other…

Ah, more problems with shell expansion and substitution. Pipelines to the rescue. If you used a pipeline instead of shell expansion, it can deal with spaces just peachy. $ find . -name "*abc*" -print0 | xargs -0 ls -l There's no argument that bash (and other shells) make dealing with spaces and quotes troublesome. That has nothing to do with pipelines.

Does every unix utility suport -print0 or similar?

My point wasn't specifically about find, but about the problems of text and other unstructured input/output in general.

Update: reading a bit around the internet, it looks like it's not possible to do something like this :

    find / -print0 | \
        grep something | \
        xargs -0 cat
As grep will mangle the output from find -print0.

Re: The Beauty of Unix Pipelines

#354

Earlier quoted context omitted.

> because spaces are a reasonable thing to put in a filename unless you're on a Unix system. Putting spaces on a filename is atrocious and should be disallowed by modern filesystems. It is like if you could put spaces inside variable names in Python. Ridiculous.

I'm not going back to CamelCase or underscores for my normal day to day file naming. The problem with spaces only exists inside the IT world and it's something they should find a way around.

At the point I'd be happy if all unix tools had a `--json` option. Then the whole space issue goes away. So do a bunch of parsing issues.

Unix tools standardized a character representation (e.g. everything is ascii). The next step is a unified standard the syntactical representation.

Re: The Beauty of Unix Pipelines

#355

Earlier quoted context omitted.

> because spaces are a reasonable thing to put in a filename unless you're on a Unix system. Putting spaces on a filename is atrocious and should be disallowed by modern filesystems. It is like if you could put spaces inside variable names in Python. Ridiculous.

You should take a stroll in the real world sometime, where spaces and Unicode exists :)

The parent comment is extreme, and the real world is indeed very diverse, but I would also be quite surprised to find a software project with spaces in internal filenames.

Re: The Beauty of Unix Pipelines

#356

Earlier quoted context omitted.

Indeed, if it wasn’t for its reliance on .NET, it looks like PowerShell could be a massive improvement over Unix shell.

There are projects like NuShell that are similar to PowerShell but don't rely on .NET.

There is nothing like PowerShell.

Re: The Beauty of Unix Pipelines

#357

Earlier quoted context omitted.

There are projects like NuShell that are similar to PowerShell but don't rely on .NET.

There is nothing like PowerShell.

I agree that all of them so far seem to be lacking in many ways compared to PowerShell, but saying they are "nothing like" PowerShell isn't a very accurate statement in my opinion.

Re: The Beauty of Unix Pipelines

#358

Earlier quoted context omitted.

There is nothing like PowerShell.

I agree that all of them so far seem to be lacking in many ways compared to PowerShell, but saying they are "nothing like" PowerShell isn't a very accurate statement in my opinion.

No, that is totally accurate - there is nothing even close to PowerShell.

You were inaccurate on the other hand - what is similar to PowerShell ?

Re: The Beauty of Unix Pipelines

#359

Earlier quoted context omitted.

I'm not going back to CamelCase or underscores for my normal day to day file naming. The problem with spaces only exists inside the IT world and it's something they should find a way around.

At the point I'd be happy if all unix tools had a `--json` option. Then the whole space issue goes away. So do a bunch of parsing issues. Unix tools standardized a character representation (e.g. everything is ascii). The next step is a unified standard the syntactical representation.

ASCII isn’t just a “character representation” though. It includes code points for data serialisation and transmission control. You can represent tables in ASCII without resorting to printable characters nor white space as field separators.

I don’t why POSIX tools don’t already do this but I suspect it’s because most tools would have evolved by accident rather than being designed from the ground up with edge cases in mind.

Re: The Beauty of Unix Pipelines

#360

Earlier quoted context omitted.

Me too. Especially since I can regularly hit 100 WPM when typing, but I'm a terrible shot with the mouse - not to mention the fact that you can get really nice and satisfying keyboards (I use an IBM Model M at home and a CODE Cherry MX Clear at college) but mice are all kind of the same, and you have to move your hand a lot to get there. On that last point, my mouse hand gets wrist pain consistently way more than my…

Ditto. For projects where I must work in Windows my secret weapon is WSL + tmux + fish + vim for everything without a strict Visual Studio dependency.

hard to get around not using the mouse eventually when looking up docs or copy pasting from browser
Post reply on HN