Live data from Hacker News

Bash code generator for command-line arguments

github.com

41–50 of 54 posts

Re: Bash code generator for command-line arguments

#41
post #28

Earlier quoted context omitted.

This is nice! I suggest writing messages to standard error instead of standard output.

You mean like this? echo "message" 1>&2 Yes, that's probably a good idea, though I cannot recall this causing a problem for any script that I've written so far.

[deleted]

Re: Bash code generator for command-line arguments

#42
post #28

Earlier quoted context omitted.

This is nice! I suggest writing messages to standard error instead of standard output.

You mean like this? echo "message" 1>&2 Yes, that's probably a good idea, though I cannot recall this causing a problem for any script that I've written so far.

Wait, --help should print to stdout so that

    fancycommand --help | less
works as expected. Otherwise, the help text will dump to the screen and less will erase it when it decides to redraw its empty buffer! Actual error messages should go to stderr so that I can redirect them to an error log or so I can suppress them if need be, or I could also be doing > /dev/stdout to silence normal output but I don't want errors to be discarded.

Re: Bash code generator for command-line arguments

#43
post #20

I start all my bash scripts from the following template, then modify it as needed. It is not fancy, but I think it supports the lowest common denominator that is compatible with other flag processing systems in all other languages that I am aware of. (In particular, it does not support '--option=opt' but uses '--option opt', which is cross-compatible). I like that it is small, easy to maintain, and avoids external de…

I recommend making 'usage' return 0, rather than 1; since asking for help is valid. (Since you use it both for -h and for unknown options, perhaps make it not exit at all, and make the caller exit with an appropriate code?) Additionally, you should output the actual executable name (as passed in on $0) rather than hardcoding something. So maybe at the top say 'executable=$0; shift'; replace parseflags.sh with $execut…

I think technically you are correct about --help being valid. But in practice, I use --help only in an interactive context where the exit code does not matter so I've never been bitten by the subtle difference.

I tried using $0 years ago, but I didn't like seeing the full path name (or the "./parseflags.sh" string), or maybe it was because I often use shell alias, so the $0 becomes the alias name, or something like that. So now, I just replace the string "parseflags.sh" with the name of the script for each script. I think I tried using $(basename $0) at one point, but didn't like that either, though I don't remember why.

I hope that the template is simple enough that people can customize it as they wish.

Edit: Fix typo

Re: Bash code generator for command-line arguments

#44
Argbash is great. I found the output unnecessarily verbose for the simple functionality that I wanted, so the way I've used it has been to put the following in a script/function and then eval the output: https://gist.github.com/buu700/72d0461d318bfe8c11e36d2316882...

That would then be used like:

    eval "$(parseArgs \
        --opt foo \
        --opt-bool bar \
        --pos balls \
    )"`

Re: Bash code generator for command-line arguments

#45
Yeah this problem with shell is real ... it ironically is not good at writing command line interfaces!

If anyone is interested in Oil, I would appreciate help either:

(1) Running this library with OSH. Try it and tell me what breaks! OSH runs many unmodified bash scripts.

(2) Help with a new builtin that will address the deficiencies of getopt/getopts. You could help design it!

https://github.com/oilshell/oil/issues/469

Re: Bash code generator for command-line arguments

#46
post #7

Earlier quoted context omitted.

It's a trade-off. Especially if the script contains a lot of pipes and redirection, it's difficult to reproduce with the same elegance in something like Python. My biggest successes have been extracting these parts into very small stand-alone shell scripts which the Python application then invokes, which gives you the best of both worlds. The best example off the top of my head is `mysqldump | sed ... | gzip` with de…

I’m curious about your experience that pipes and redirects are harder with python. Do you mean slightly more verbose or just complicated and difficult? I’m curious if you’ve tried the sh Python module [1]. In some ways it’s much prettier to read although the initial writing of it can be awkward to get used to at first. [1] https://amoffat.github.io/sh/

I mean significantly more verbose and more complicated. Fee free to try it yourself - replicate `bash -eo pipefail -c 'mysqldump example_db | gzip > dump.sql.gz'` in Python with streaming (databases can be several GB, so you can't read it all in memory).

I have not tried any of these libraries, though they look nice. The only times I've had to write scripts that make heavy use of subprocesses and pipes, I want them to work with only the standard library so I can just rsync them and they just work™.

Re: Bash code generator for command-line arguments

#47
post #20

I start all my bash scripts from the following template, then modify it as needed. It is not fancy, but I think it supports the lowest common denominator that is compatible with other flag processing systems in all other languages that I am aware of. (In particular, it does not support '--option=opt' but uses '--option opt', which is cross-compatible). I like that it is small, easy to maintain, and avoids external de…

I recommend making 'usage' return 0, rather than 1; since asking for help is valid. (Since you use it both for -h and for unknown options, perhaps make it not exit at all, and make the caller exit with an appropriate code?) Additionally, you should output the actual executable name (as passed in on $0) rather than hardcoding something. So maybe at the top say 'executable=$0; shift'; replace parseflags.sh with $execut…

just fyi, `shift` doesn't affect $0, it slides the window of $1..$N back, so you don't want to shift after grabbing $0 and you don't want to change the option loop if you do this.

And you can just use $0 everywhere so there's technically no reason to pull it out into a $executable variable, but tbh that gets a bit confusing in functions.

Re: Bash code generator for command-line arguments

#48
There's a bit of code duplication[0] that often emerges as a result of parsing command-line arguments in shell scripts. The duplication can be eliminated by defining[1] the set of arguments to pass into a reusable template script[2].

[0]: https://dave.autonoma.ca/blog/2019/06/16/typesetting-markdow...

[1]: https://github.com/DaveJarvis/keenwrite/blob/master/installe...

[2]: https://github.com/DaveJarvis/keenwrite/blob/master/build-te...

Re: Bash code generator for command-line arguments

#49
post #19
post #11

> getopt is discouraged By who? Why? This is the version that I’ve always used: https://manpages.debian.org/buster/util-linux/getopt.1.en.ht...

Start at https://unix.stackexchange.com/q/62950/5132 .

Summary of that reference: If you’re on Linux, then you’re fine, and getopt(1) is good.

Re: Bash code generator for command-line arguments

#50

Earlier quoted context omitted.

I’m curious about your experience that pipes and redirects are harder with python. Do you mean slightly more verbose or just complicated and difficult? I’m curious if you’ve tried the sh Python module [1]. In some ways it’s much prettier to read although the initial writing of it can be awkward to get used to at first. [1] https://amoffat.github.io/sh/

I mean significantly more verbose and more complicated. Fee free to try it yourself - replicate `bash -eo pipefail -c 'mysqldump example_db | gzip > dump.sql.gz'` in Python with streaming (databases can be several GB, so you can't read it all in memory). I have not tried any of these libraries, though they look nice. The only times I've had to write scripts that make heavy use of subprocesses and pipes, I want them t…

You have to dig through the documentation a bit on the website but all the info is there[1]

  from sh import mysqldump
  from sh import gzip

  gzip(mysqldump("example_db", _piped=True), _out="dump.sql.gz")
or if you don't want the magic import thing:

  import sh
  sh.gzip(sh.mysqldump("example_db", _piped=True), _out="dump.sql.gz")
It's definitely foreign to shell scripting languages since the piping syntax is a bit different & you have to remember to do `_piped=True` since it's not parallel by default. But the default behavior is pipefail & exit on the command failing IIRC so it's more of a choose your poison thing (do you want a subtle perf issue or a subtle bug in your script not handling error states correctly). And I find it easier to read. + if you want to customize anything about gzip or not rely on needing the binary in the path, then you can just switch it to Python-native gzip pretty easily.

That's my favorite feature. Conciseness if I'm just translating a script with progressive complexity options to migrate things that need more complexity or different requirements within the same script without having to rewrite it from scratch.

[1] https://amoffat.github.io/sh/

Post reply on HN