Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Bash Completion Works

tuzz.tech

Post reply on HN
Print view View original

41–49 of 49 posts

Re: How Bash Completion Works

#41

Post by vunie »

shakna wrote: post #33
vunie wrote: post #21

My issue with bash completion is that it requires a completion script (i.e executing `complete`) for each command you want it to complete. The shell cannot automatically deduce appropriate completions when possible. This problem is not specific to bash. Fish and other shells can't automatically complete commands either. There is simply no standard way to detect what type of auto-completion a command supports. I know…

> The shell cannot automatically deduce appropriate completions when possible. I don't think this even _can_ be possible for a large number of programs. For some small number of programs it _might_ be possible, if somehow (ignoring how for now) you were exposed some standard interfaces like getopt or argparse and didn't have to care about argument ordering. But a ton of other programs, with extremely complicated inte…

You raise a good point about solvable completion, and one that I agree with. However, automatic completion support doesn't have to cover every use case. It only needs to be good enough. Complex applications can fall back to providing their own completion scripts.

One point I'm not clear on is why you think such a scheme wouldn't be feasible for a large number of commands. Completion wouldn't be invoked until the fist argument (the command's name) is complete just like how bash doesn't scan the system's completion directory until it knows what command is being invoked.

And as for completion for the command's arguments, the command is only ran once to generate the list of completion candidates.

Note that there are projects that already do exactly this like [1] .

[1]:https://github.com/kislyuk/argcomplete

Re: How Bash Completion Works

#42

Post by shakna »

vunie wrote: post #41
shakna wrote: post #33

Earlier quoted context omitted.

> The shell cannot automatically deduce appropriate completions when possible. I don't think this even _can_ be possible for a large number of programs. For some small number of programs it _might_ be possible, if somehow (ignoring how for now) you were exposed some standard interfaces like getopt or argparse and didn't have to care about argument ordering. But a ton of other programs, with extremely complicated inte…

You raise a good point about solvable completion, and one that I agree with. However, automatic completion support doesn't have to cover every use case. It only needs to be good enough. Complex applications can fall back to providing their own completion scripts. One point I'm not clear on is why you think such a scheme wouldn't be feasible for a large number of commands. Completion wouldn't be invoked until the fist…

> One point I'm not clear on is why you think such a scheme wouldn't be feasible for a large number of commands. Completion wouldn't be invoked until the fist argument (the command's name) is complete just like how bash doesn't scan the system's completion directory until it knows what command is being invoked.

Simply because some command pipelines are extremely complicated. For example, with ffmpeg, some flags can disable earlier flags, or re-enable them. Completion isn't a simple left-to-right thing. There's other programs that can be similar.

Re: How Bash Completion Works

#43

Post by jolmg »

opk wrote: post #30
earenndil wrote: post #29

Earlier quoted context omitted.

At least in zsh, you can interrupt slow autocompletes with ^C.

Zsh also lets you configure, via the `remote-access` style whether you want to allow it to make remote connections for completion to begin with.

My experience actually is with zsh, it is about remote-access, and I cannot interrupt it with Ctrl-C.

The particular case is normal filepath completion on an NFS mount where the NFS server is a very old Unix OS using a very old version of the NFS protocol. The thing is that while the server hangs under particular circumstances, and zsh is trying to read the directory to autocomplete paths, the kernel cannot return from the normal FS system calls that are invoked, and the process gets stuck in uninterruptible IO. Not even SIGKILL works then.

Re: How Bash Completion Works

#44

Post by vunie »

shakna wrote: post #42
vunie wrote: post #41

Earlier quoted context omitted.

You raise a good point about solvable completion, and one that I agree with. However, automatic completion support doesn't have to cover every use case. It only needs to be good enough. Complex applications can fall back to providing their own completion scripts. One point I'm not clear on is why you think such a scheme wouldn't be feasible for a large number of commands. Completion wouldn't be invoked until the fist…

> One point I'm not clear on is why you think such a scheme wouldn't be feasible for a large number of commands. Completion wouldn't be invoked until the fist argument (the command's name) is complete just like how bash doesn't scan the system's completion directory until it knows what command is being invoked. Simply because some command pipelines are extremely complicated. For example, with ffmpeg, some flags can d…

> Completion isn't a simple left-to-right thing.

Sorry, I still don't understand why that's relavent. It's probably my fault for not being clear enough. Please be patient with me as I try to further elaborate:

If the command is in charge of both generating completions and also for parsing arguments, wouldn't it be aware of all inner-dependencies between flags? Wouldn't such awareness make it possible to generate candidates based on previous flags?

I'm not familiar with ffmpeg, but let's assume that there was another similar program called ggmpeg. Let's assume that it accepts the flag '--input-format' and then either the flag '--repeat-video-range' or '--repeat-image-frames', depending on whether the format is an video or an image.

That is, only the following flag sequences are valid:

    ggmpeg --input-format=video/webm --repeat-video-range=.. ...
or

    ggmpeg --input-format=image/png --repeat-image-frames=.. ...
Let's also assume that ggmpeg signals to the shell that it is responsible for generating its own completions by passing the flag --_completion to it.

Now, if the user types the following command

    ggmpeg --input-format=image/png --repea[CURSOR]
and then presses tab, thereby invoking the shell's auto completion routine, which in turn ultimately executes

    "ggmpeg" "--_completion" "--input-format=image/png" "--repea"
wouldn't ggmpeg have all the necessary information to only suggest '--repeat-image-frames' and not '--repeat-video-range'?

Re: How Bash Completion Works

#45

Post by shakna »

vunie wrote: post #44
shakna wrote: post #42

Earlier quoted context omitted.

> One point I'm not clear on is why you think such a scheme wouldn't be feasible for a large number of commands. Completion wouldn't be invoked until the fist argument (the command's name) is complete just like how bash doesn't scan the system's completion directory until it knows what command is being invoked. Simply because some command pipelines are extremely complicated. For example, with ffmpeg, some flags can d…

> Completion isn't a simple left-to-right thing. Sorry, I still don't understand why that's relavent. It's probably my fault for not being clear enough. Please be patient with me as I try to further elaborate: If the command is in charge of both generating completions and also for parsing arguments, wouldn't it be aware of all inner-dependencies between flags? Wouldn't such awareness make it possible to generate cand…

Ok, here's a simpler example that might demonstrate why it can't always be solved.

    ffmpeg -i input.avi -metadata author=shakna output.mp4
What metadata keys are allowed, if at all, actually depends on the output file. If I'd chosen to make an ogv instead, then that metadata flag would become disabled, because it doesn't accept most sets of metadata.

I can't complete:

    ffmpeg -i input.avi -metadata a[TAB]
Because the entire set of CLI commands isn't simple left-to-right solveable.

However, if I do:

    ffmpeg -i input.avi -metadata author=shakna output.mp4 output.ogv
That means I'm outputting one file with metadata, output.mp4, and one file without it, output.ogv. So the flag is conditionally enabled or disabled.

Re: How Bash Completion Works

#46

Post by hnlmorg »

vunie wrote: post #40
hnlmorg wrote: post #39

Earlier quoted context omitted.

Some shells do parse manages and use other tricks to auto-determine completes. Though they're not POSIX compliant shells so you'd have a to retrain a little there. I'm interested to know more about that ELF / --_complete trick though. Do you have any more details? (when DDging the only result I can find on the topic is your comment)

> (when DDging the only result I can find on the topic is your comment) You weren't kidding at all. After spending more time than I'd care to admit searching for the library, I couldn't find traces of it anywhere on google, ddg, or bing. I relented and digged whatever information I could find in my history. Revisiting these projects, it's painfully obvious that I have jumbled a few projects together. Sincere apologie…

Awesome. I really appreciate you taking the time to research that for me. Some great material for me to read through. Thank you

Re: How Bash Completion Works

#47

Post by vunie »

shakna wrote: post #45
vunie wrote: post #44

Earlier quoted context omitted.

> Completion isn't a simple left-to-right thing. Sorry, I still don't understand why that's relavent. It's probably my fault for not being clear enough. Please be patient with me as I try to further elaborate: If the command is in charge of both generating completions and also for parsing arguments, wouldn't it be aware of all inner-dependencies between flags? Wouldn't such awareness make it possible to generate cand…

Ok, here's a simpler example that might demonstrate why it can't always be solved. ffmpeg -i input.avi -metadata author=shakna output.mp4 What metadata keys are allowed, if at all, actually depends on the output file. If I'd chosen to make an ogv instead, then that metadata flag would become disabled, because it doesn't accept most sets of metadata. I can't complete: ffmpeg -i input.avi -metadata a[TAB] Because the e…

I can now see what you meant by completion not being left-to-right solvable. Thanks for taking the time to lay out a clear example.

While completion isn't be solvable in general, surly we can agree that its usefulness (when solvable) merits making it more readily available to users.

Having the shell implementation automatically deduce suggestions from the command itself (when available/possible) would go a long way towards increasing availability in my opinion.

Re: How Bash Completion Works

#48

Post by p_ronto »

karatinversion wrote: post #8

Nice article, it’s usually hard to keep a reader’s interest when writing about shell. I will nitpick that COMP_LINE et al are shell variables, not environment variables - you can tell by the fact you don’t need to export the “return value”. (Scare quotes, as the function’s exit code, manually specifiable with “return NN”, is also called its return value.)

I think they are env vars. complete is a bash built-in. It is executed in the process of the shell. You only need to export for child processes.

Re: How Bash Completion Works

#49

Post by opk »

jolmg wrote: post #43
opk wrote: post #30

Earlier quoted context omitted.

Zsh also lets you configure, via the `remote-access` style whether you want to allow it to make remote connections for completion to begin with.

My experience actually is with zsh, it is about remote-access, and I cannot interrupt it with Ctrl-C. The particular case is normal filepath completion on an NFS mount where the NFS server is a very old Unix OS using a very old version of the NFS protocol. The thing is that while the server hangs under particular circumstances, and zsh is trying to read the directory to autocomplete paths, the kernel cannot return fr…

There's a good chance that setting the path-completion style to false will help in this case. That does disable a useful feature - though it is only a useful feature if you know it is there and use it. If not even SIGKILL works, it is stuck in kernel code. That doesn't surprise me much with NFS. That's not easy to workaround in the zsh code - other than by disabling functionality as with my path-completion suggestion.
Post reply on HN