Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hints for writing Unix tools

monkey.org

Post reply on HN
Print view View original

71–80 of 131 posts

Re: Hints for writing Unix tools

#71

Post by Someone »

foobarbaz1234 wrote: post #68
osandov wrote: post #25

A nitpicky tip: --help is normal execution, not an error, so the usage information should be printed to stdout, not stderr (and it should exit with a successful status). Nothing is more annoying than trying to use a convoluted program with a million flags (which should have a man page in the first place) and piping --help into less with no success.

I am not so sure with that. Say, your program is used in a shell script and is invoked badly - you might want to print its usage then. If you exit normally your shell script might break weirdly but if you exit with error it's easier to spot the reason of failure. On the other hand you made me thinking and probably you should have three code passes per default: [0] normal behaviour (exit 0) [1] bad arguments (exit EIN…

In my book, there is a difference between explicitly asking for help/usage and passing arguments that do not make sense, which triggers the output of help/usage.

The former, I think, should write to stdout and return 0, the latter should write to stderr and return something non-zero.

Giving help if the user asks for it is normal behaviour.

Re: Hints for writing Unix tools

#72

Post by dec0dedab0de »

ygra wrote: post #53
voltagex_ wrote: post #2

I'm not sure I agree with the "no JSON, please" remark. If I'm parsing normal *nix output I'm going to have to use sed, grep, awk, cut or whatever and the invocation is probably going to be different for each tool. If it's JSON and I know what object I want, I just have to pipe to something like jq [1]. PowerShell takes this further and uses the concept of passing objects around - so I can do things like ls | $_.Name…

I was also constantly thinking of PowerShell while reading that. A PowerShell-specific list of such advice would actually be rather short, given that most of the pitfalls are already avoided. I still firmly believe that PowerShell is actually a much more consistent Unix shell in that several concepts that ought to be separate are actually orthogonal. Let's see: Input from stdin, output to stdout: Nicely side-stepped…

I have never used a computer that had access to Powershell, but in my new job I may have to do some small stuff to tie some systems together. I'm terrified of learning it because I don't want to be lured into some kind of lock-in scenario.

Re: Hints for writing Unix tools

#73

Post by philh »

chriswarbo wrote: post #63
burke wrote: post #28

Earlier quoted context omitted.

I think it depends what sort of things you use it for. I often use it to switch on or off ANSI colourization, which doesn't really violate the principle of least surprise. When used sparingly and thoughtfully, I've never personally had an issue with it.

You may not have issue with the sort of things you use it for, but others might. For example, I run shells in Emacs and have had to tweak loads of shell scripts written by colleagues to fix their poorly-implemented colourisation. It's useful to know when a test has failed; it's not so useful to have the whole terminal set to white text on a pale pink background. One day I couldn't SSH into our servers from Emacs. It…

Are you suggesting that colored prompts violate the rules of consistency and least surprise?

(Actually, if you are suggesting that, I'm not going to disagree. But I am going to say that if so, those rules don't apply in the case of colored prompts, because colored prompts are useful.)

Re: Hints for writing Unix tools

#74

Post by bmn_ »

deathanatos wrote: post #23
pstuart wrote: post #15

Earlier quoted context omitted.

> That breaks when you have newlines in filenames, no? That seems like an extremely pathological case.

> That seems like an extremely pathological case. When a human is creating files by hand, I almost certainly agree. When a program is creating files, however, it's only a matter of time before weird characters wind their way in there. I really wish newlines had been disallowed. (There's UI implications, in addition to the parsing ones — how do you do a list view with newlines in the filename?; I also wish filenames h…

I think dwheeler is trying to get this fixed/standardised in POSIX via the Open group.

Re: Hints for writing Unix tools

#75

Post by chriswarbo »

philh wrote: post #73
chriswarbo wrote: post #63

Earlier quoted context omitted.

You may not have issue with the sort of things you use it for, but others might. For example, I run shells in Emacs and have had to tweak loads of shell scripts written by colleagues to fix their poorly-implemented colourisation. It's useful to know when a test has failed; it's not so useful to have the whole terminal set to white text on a pale pink background. One day I couldn't SSH into our servers from Emacs. It…

Are you suggesting that colored prompts violate the rules of consistency and least surprise? (Actually, if you are suggesting that, I'm not going to disagree. But I am going to say that if so, those rules don't apply in the case of colored prompts, because colored prompts are useful .)

I suppose I'm suggesting that, aside from personal scripts, we shouldn't assume too much about who our users are and what they're trying to do. The principle of least power tells us to use the dumbest format that will work, eg. plain text.

Anything we add on top of that, eg. ANSI colour codes, will be useful to some but harmful to others. The tricky part is working out which of those categories the current user is in.

Re: Hints for writing Unix tools

#76

Post by james2vegas »

dTal wrote: post #57
acabal wrote: post #3

Great article. The other thing I've always wished for command-line tools is some kind of consistency for flags and arguments. Kind of like a HIG for the command line. I know some distros have something like this, and that it's not practical to do as many common commands evolved decades ago and changing the interface would break pretty much everything. But things like `grep -E,--extended-regexp` vs `sed -r,--regexp-ex…

I've often thought this - that xkcd.com/1168/ is funny is a terrible embarrassment. I would also like to add that manpage syntax help should be standardized and machine-parseable. I had an idea recently to auto-generate GUIs for command line tools from the manpage syntax line, but it turned out that while such lines look precise but cryptic, they are often in fact highly ambiguous, nonstandard, and still cryptic. Thi…

Blame man(7), have a look at mdoc(7): Semantic markup for command line utilities: Nm : start a SYNOPSIS block with the name of a utility; Fl : command line options (flags) (>=0 arguments); Cm : command modifier (>0 arguments); Ar : command arguments (>=0 arguments); Op, Oo, Oc : optional syntax elements (enclosure); Ic : internal or interactive command (>0 arguments); Ev : environmental variable (>0 arguments); Pa : file system path (>=0 arguments)

Re: Hints for writing Unix tools

#77

Post by InfiniteRand »

ygra wrote: post #53
voltagex_ wrote: post #2

I'm not sure I agree with the "no JSON, please" remark. If I'm parsing normal *nix output I'm going to have to use sed, grep, awk, cut or whatever and the invocation is probably going to be different for each tool. If it's JSON and I know what object I want, I just have to pipe to something like jq [1]. PowerShell takes this further and uses the concept of passing objects around - so I can do things like ls | $_.Name…

I was also constantly thinking of PowerShell while reading that. A PowerShell-specific list of such advice would actually be rather short, given that most of the pitfalls are already avoided. I still firmly believe that PowerShell is actually a much more consistent Unix shell in that several concepts that ought to be separate are actually orthogonal. Let's see: Input from stdin, output to stdout: Nicely side-stepped…

My issue with Powershell is it creates a distance from the scripting language and an ordinary executable, which makes it difficult to use just a little bit (and I suppose violates the rule about composability).

Re: Hints for writing Unix tools

#78

Post by lstamour »

tfigment wrote: post #30
lstamour wrote: post #18

Earlier quoted context omitted.

On Mac (OS NeXT, perhaps?), the convention seems to be that most commands produce human readable output by default, but you can pass a parameter like -x or -xml to get (usually) XML, machine-readable output, and with some tools, -j or -json will give you that format. But then you've oddities like plutil behaving like gzip by modifying the file you specify rather than printing to stdout. You have to pass -o and a dash…

Not sure if its what you had in mind for Windows and tabs but I've found ConsoleZ [1] quite nice and allows powershell, cmd and others to have tabs. [1] https://github.com/cbucher/console

Yeah, I know it's possible with adding, I've tried Console2 and ConEmu before. But I'd like it to just work out of the box with no extra software, as it does on Mac or Linux. Until Windows 10, that terminal hadn't changed since NT days...

Re: Hints for writing Unix tools

#79

Post by grymoire1 »

jzwinck wrote: post #12

Here's one more tip: did you ever notice that "ls" displays multiple columns, but "ls | cat" prints only one filename per line? Or how "ps -f" truncates long lines instead of wrapping, while "ps -f | cat" lets the long lines live? You can do it too, and if you're serious about writing Unix-style filter programs, you will someday need to. How do you know which format to write? Call "isatty(STDOUT_FILENO)" in C or C++,…

As I recall, the original ls didn't have that feature.

Examining the characteristics of the output stream and changing behavior is another "rule" that is not mentioned often. Another example is buffering the output to a large block if sending to a pipe, but making it line-buffered if going to a terminal.

Re: Hints for writing Unix tools

#80

Post by michaelmior »

voltagex_ wrote: post #2

I'm not sure I agree with the "no JSON, please" remark. If I'm parsing normal *nix output I'm going to have to use sed, grep, awk, cut or whatever and the invocation is probably going to be different for each tool. If it's JSON and I know what object I want, I just have to pipe to something like jq [1]. PowerShell takes this further and uses the concept of passing objects around - so I can do things like ls | $_.Name…

Yeah, I love jq. With a tool like that, I'd actually like to have an option for standard *nix tools to output JSON. Dealing with structured output would be far easier than counting which columns need to be extracted, using sed to split things, etc.
Post reply on HN