Live data from Hacker News

Hints for writing Unix tools

monkey.org

121–130 of 131 posts

Re: Hints for writing Unix tools

#121
post #90
post #84

Earlier quoted context omitted.

That's an interesting case. Should it not just output to stdout and you have to pipe it to less? If you wanted it to always pipe to less you could just set up an alias for it.

I've just realised that the best solution to this (which I've never seen) is for the shell or terminal emulator to capture long output into a pager for you once it exceeds a certain length.

I think there may be some flags in pagers to do this for you, but I can't remember for sure.

Re: Hints for writing Unix tools

#122
post #15

Earlier quoted context omitted.

> That breaks when you have newlines in filenames, no? That seems like an extremely pathological case.

> That seems like an extremely pathological case. When a human is creating files by hand, I almost certainly agree. When a program is creating files, however, it's only a matter of time before weird characters wind their way in there. I really wish newlines had been disallowed. (There's UI implications, in addition to the parsing ones — how do you do a list view with newlines in the filename?; I also wish filenames h…

how do you do a list view with newlines in the filename?

Show them with the standard escape sequence for a newline:

    This\ filename\ncontains\ a\ newline
Same for any other characters that could be considered 'special' in output; I really wish the backslash convention for escaping was more common. Character sets and such are a UI/display issue, so I don't think there should be any special handling for them at the lower levels of the system.

Re: Hints for writing Unix tools

#123
post #96

I think it's insane to restrict programs to just STDOUT & STDERR. Why 2? Why not use another file descriptor, maybe STDFMT, to capture all the formatting markup? This would avoid -0 options (newlines are markup sent to stdfmt, all strings on stdout are 0-terminated), it would avoid -H options (headers go straight to STDFMT), it would allow for less -R to still work, etc. It's possible other descriptors would be usefu…

I honestly have no idea what you are talking about. The whole point of standard i/o streams is for them to be portable and composable by other programs without those programs having to be designed to work with yours. POSIX is here for a very good reason. Obviously not every program will use just two file descriptors. Binary isn't handled by stdin and stdout because they're typically used for tty input/output. If you…

> In your program's design, the 'cat' program would handle all kinds of file i/o, provide some kind of ncurses text GUI to select a file, a progress bar for the progress of text flowing through it, sending errors to a logging subsystem

Not at all. cat wouldn't have a ncurses GUI, that doesn't make sense. My point is that 'cat --verbose' should be an option, where the stdout doesn't change but extra crap is sent elsewhere, and probably just dumped on the terminal like stderr. I sometimes want to see extra context and line numbers in my grep searches (grep -nC 3 ..) but I might want the stdout to remain clean. This makes programs more composable. Right now it's like we've got stdfmt permanently redirected towards stdout.

In practical terms, vi does its own paging. It's not a wrapper over echo | ed | less. One giant monolithic subsystem. Perhaps vi is the exception. But dd offers a progress bar, but only if you send it a SIG of some sort. wget offers a progress bar by default (silence is golden? not so much). ls yields differently columned outputs to ttys or files. I suppose this is the simplicity of Unix that I shouldn't touch.

Some unix tools work really well already, and I'm not suggesting destroying tar or xargs. I'm not sure how systemd works into this, but I'm not really a fan of that.

I guess Plan9 wasn't Unix, either.

Re: Hints for writing Unix tools

#126
post #111

Earlier quoted context omitted.

-w

It doesn't say that it'll do that automatically when run through a pipe.

Yep, and the result is I have needless sprinklings of "www" in my shell scripts cause of habit. Technically I don't need it the moment I pipe into grep, but oh well ;) Anyway I personally dislike the SysV-like 'I need to add stuff like -aef' to get all that on-the-screen/into-grep vs the BSD-like it knows about TTY but I can convince it otherwise into stuf like 'less -S' - personal taste I guess.

Re: Hints for writing Unix tools

#127
post #126

Earlier quoted context omitted.

It doesn't say that it'll do that automatically when run through a pipe.

Yep, and the result is I have needless sprinklings of "www" in my shell scripts cause of habit. Technically I don't need it the moment I pipe into grep, but oh well ;) Anyway I personally dislike the SysV-like 'I need to add stuff like -aef' to get all that on-the-screen/into-grep vs the BSD-like it knows about TTY but I can convince it otherwise into stuf like 'less -S' - personal taste I guess.

Actually I just found this in the 'ps' manual, it looks like the output width is actually undefined! "If ps can not determine display width, as when output is redirected (piped) into a file or another command, the output width is undefined (it may be 80, unlimited, determined by the TERM variable, and so on)."

Re: Hints for writing Unix tools

#128

Earlier quoted context omitted.

> That seems like an extremely pathological case. When a human is creating files by hand, I almost certainly agree. When a program is creating files, however, it's only a matter of time before weird characters wind their way in there. I really wish newlines had been disallowed. (There's UI implications, in addition to the parsing ones — how do you do a list view with newlines in the filename?; I also wish filenames h…

how do you do a list view with newlines in the filename? Show them with the standard escape sequence for a newline: This\ filename\ncontains\ a\ newline Same for any other characters that could be considered 'special' in output; I really wish the backslash convention for escaping was more common. Character sets and such are a UI/display issue, so I don't think there should be any special handling for them at the lowe…

[deleted]

Re: Hints for writing Unix tools

#129
post #7

Additional tip: if writing a tool that prints a list of file names, provide a -0 option that prints them separated by '\x0' rather than white space. Then the output can be piped through xargs -0 and it won't go wrong if there are files with spaces in their paths. I suggest -0 for symmetry with xargs. find calls it -print0, I think. (In my view, this is poor design on xargs's part; it should be reading a newline-separ…

You view was heard in the design of GNU Parallel: It defaults to newline separation, escapes the argument, and is for most cases a drop-in replacement of xargs.

This does what you would expect:

  echo My brother\'s 12\" records.txt | parallel touch

Re: Hints for writing Unix tools

#130
post #84

Earlier quoted context omitted.

And yet .. programs are not just for composition. They have to behave sensibly for people. I love that 'git log' outputs in a pager. 'svn log' by comparison is nuts.

That's an interesting case. Should it not just output to stdout and you have to pipe it to less? If you wanted it to always pipe to less you could just set up an alias for it.

I believe software should be written to to satisfy the common case and the common case is the non-expert.
Post reply on HN