Live data from Hacker News

Use Long Options in Scripts

matklad.github.io

111–120 of 157 posts

Re: Use Long Options in Scripts

#111

Reminds me of our code having llm generated regular expressions which are impossible to understand and the only way you can tweak it is giving it to llm to change.

Folks, remember record your LLM prompt in a comment so that your regex can be validated.

Re: Use Long Options in Scripts

#112
post #95
post #88

Before invoking a command, always first check if the length of the command is not longer than ARG_MAX. For example, if this is your command: grep --ignore-case --files-with-matches -- "hello" *.c Then invoke it as follows: CMD="grep --ignore-case --files-with-matches -- \"hello\" *.c" ARG_MAX=$(getconf ARG_MAX) CMD_LEN=${#CMD} if (( CMD_LEN > ARG_MAX )); then echo "Error: Command length ($CMD_LEN) exceeds ARG_MAX ($A…

That might be sensible, but also obscure the script logic. Since using Linux exclusively, I don't think I've ever encountered an issue due to too many arguments/length. And it's the first time I'm actively searching online for ARG_MAX. I understand that different shells might be different, but with reasonable lengths is there any chance of it being relevant (aside from xargs, where it's generally intended, or better,…

I started running into these issues when I started working with training examples in the context of deep learning. A folder with millions of files is then not unheard of.

Also if you do things like */*/*, then you can quickly get large command lines. Or even if you do long_name/another_long_name/*.

Re: Use Long Options in Scripts

#113
post #84

I prefer long options too. However, while writing programs that need to invoke POSIX commands in a portable manner, short options are the only viable choice, as POSIX doesn't specify long options. For instance, see the specification for diff at https://pubs.opengroup.org/onlinepubs/9799919799/utilities/d... >, or that of any POSIX utility listed at https://pubs.opengroup.org/onlinepubs/9799919799/idx/utiliti... >. Th…

> However, while writing programs that need to invoke POSIX commands in a portable manner ...probably a stupid question, but something I have earnestly been wondering about... when does this actually happen nowadays? What POSIX systems are you targeting that aren't one of the major ones (Linux, Darwin, or one of the major BSDs)? I was writing a shell script a few months ago that I wanted to be very durable, and I tar…

I ~recently had to wrestle w/ TeamCity (CICD) whose build agents provide only sh. I needed to invoke a 3rd-party util that required bash. The resulting "bash in dash in docker in docker" worked, but I wasn't thrilled about the convoluted / Frankenstein setup.

Re: Use Long Options in Scripts

#115
post #105
post #96

Earlier quoted context omitted.

You should always type check your shell scripts as well. For example, you just: $ shelltypes script.sh # Welcome to shelltypes v 3.23.2 # type ‘help’ if you’re stuck >>> {1) # import POSIX;; Importing 73 items. >>> {2} # append loadpath “/opt/local/shelltypes/base”;; >>> {3} # import base::YOURPROJECT;; Importing 15 items. >>> {4} # check “YOURSCRIPT.sh” Parsing YOURSCRIPT.sh. Reticulating splines. Expanding aliases.…

Where does "shelltypes" come from? I can't find anything on DuckDuckGo or Google, but this seems like it would be very useful.

From the output in the post I'm going to assume it's either a joke post or an LLM hallucination.

Re: Use Long Options in Scripts

#116

Earlier quoted context omitted.

On my system, `getconf ARG_MAX` is over 2m. I have seen some heinously long cmdline strings, but nothing close to that. Usually when invocations have creeped up into the O(1k) character limit at places I've worked, I've implemented a "yaml as cmdline args" option to just pass a config file instead. Have you seen scenarios where this is actually limiting?

Yes, if *.c expands to a string over 2m. Maybe that is a lot for .c files, but it may easily happen with .tiff and a folder full of images used for training a deep learning model, for example.

Thanks, this is interesting. I have done a lot of this sorta stuff (glob expanding giant directories) without a thought for this `ARG_MAX` libc parameter, but now I know I need to keep it in mind!

Re: Use Long Options in Scripts

#117
post #105

Earlier quoted context omitted.

Where does "shelltypes" come from? I can't find anything on DuckDuckGo or Google, but this seems like it would be very useful.

From the output in the post I'm going to assume it's either a joke post or an LLM hallucination.

Oh god, it even says right there:

> Warning: this utility is fake.

Well played! I guess I got too excited at the possibility of such a tool existing.

Re: Use Long Options in Scripts

#118
post #84

I prefer long options too. However, while writing programs that need to invoke POSIX commands in a portable manner, short options are the only viable choice, as POSIX doesn't specify long options. For instance, see the specification for diff at https://pubs.opengroup.org/onlinepubs/9799919799/utilities/d... >, or that of any POSIX utility listed at https://pubs.opengroup.org/onlinepubs/9799919799/idx/utiliti... >. Th…

> However, while writing programs that need to invoke POSIX commands in a portable manner ...probably a stupid question, but something I have earnestly been wondering about... when does this actually happen nowadays? What POSIX systems are you targeting that aren't one of the major ones (Linux, Darwin, or one of the major BSDs)? I was writing a shell script a few months ago that I wanted to be very durable, and I tar…

Alpine docker images only come with dash instead of bash, which _may_ run your sh script, but test thoroughly. Or just install bash.

FWIW, Darwin/macOS is especially guilty of gobsmackingly ancient coreutils that don’t support long option variants.

Re: Use Long Options in Scripts

#120
post #52

Earlier quoted context omitted.

I know to do this intuitively, but I have no idea why.

It terminates argument parsing, so anything following it that starts with a hyphen will not be treated as an argument. $ echo 'hack the planet' > --help $ cat --help cat: illegal option -- - usage: cat [-belnstuv] [file ...] $ cat -- --help hack the planet $ rm -vf --help rm: illegal option -- - usage: rm [-f | -i] [-dIPRrvWx] file ... unlink [--] file $ rm -vf -- --help --help $ cat -- --help cat: --help: No such fi…

My eyes have been opened. Thank you!
Post reply on HN