Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Zsh-autoquoter makes shell quoting slightly less annoying

ianthehenry.com

Post reply on HN
Print view View original

21–27 of 27 posts

Re: Zsh-autoquoter makes shell quoting slightly less annoying

#21

Post by a1369209993 »

haroldp wrote: post #8
mmastrac wrote: post #3

The shell quoting rabbithole goes pretty deep, especially when you want to preserve the whitespace within arguments. Has this been solved generally yet? I suspect it requires full knowledge of whatever shell is running on the other side of SSH.

Just thinking of one of OP's examples, his (very cool) autoquoter allows him to type: > ssh user@host awk '{print $1}' file.txt And have the "$1" quoted or escaped so that it's interpreted as an awk keyword on the remote system. But what if I actually want it interpreted as a shell variable that should be expanded on the local or remote system? How does it know? Why is shell quoting still so hard?

  > > ssh user@host awk '{print $1}' file.txt
> it's interpreted as an awk keyword on the remote system.

That's the behaviour in bash (and I think pretty much every other vaguely Bourne-like shell). That's what single quotes do.

> what if I actually want it interpreted as a shell variable that should be expanded on the local [...] system?

  $ ssh user@host awk "{print $1}" file.txt
But note that `$1` will be used as awk code, not string data. You need somthing like `\"$(echo $1 | s!(\W)!\\$1!g)\"`, the details of which will be shell-specific, if you want it to be awk code for a string constant.

> what if I actually want it interpreted as a shell variable that should be expanded on the [...] remote system?

  $ ssh user@host sh -c 'awk "{print $1}" file.txt'
Where `sh` is whatever shell you're using and `$1` specifically is obviously not very useful here.

Re: Zsh-autoquoter makes shell quoting slightly less annoying

#22

Post by andmis »

Try fish. The quoting rules are simple. I never have to read the docs or try repeatedly to get quoting right. The shell syntax in general is small and simple.

The only downside to fish is that it's not POSIX compliant, but if you're familiar with POSIX shells this is rarely a problem for interactive use, and you can use Bash for scripting if you like.

Re: Zsh-autoquoter makes shell quoting slightly less annoying

#24

Post by bool3max »

bool3max wrote: post #7
mmastrac wrote: post #3

The shell quoting rabbithole goes pretty deep, especially when you want to preserve the whitespace within arguments. Has this been solved generally yet? I suspect it requires full knowledge of whatever shell is running on the other side of SSH.

It is part of the reason why I use fish whenever possible.

String escaping in fish is phenomenal. Painless. You don't have to think about it at all. A single variable expansion results in exactly a single command line argument. As it should. As is logical.

Re: Zsh-autoquoter makes shell quoting slightly less annoying

#25

Post by haroldp »

a1369209993 wrote: post #21
haroldp wrote: post #8

Earlier quoted context omitted.

Just thinking of one of OP's examples, his (very cool) autoquoter allows him to type: > ssh user@host awk '{print $1}' file.txt And have the "$1" quoted or escaped so that it's interpreted as an awk keyword on the remote system. But what if I actually want it interpreted as a shell variable that should be expanded on the local or remote system? How does it know? Why is shell quoting still so hard?

> > ssh user@host awk '{print $1}' file.txt > it's interpreted as an awk keyword on the remote system. That's the behaviour in bash (and I think pretty much every other vaguely Bourne-like shell). That's what single quotes do . > what if I actually want it interpreted as a shell variable that should be expanded on the local [...] system? $ ssh user@host awk "{print $1}" file.txt But note that `$1` will be used as awk…

I mean, yes I know how to quote and escape. My point was that only I know what my intent was. Do I want local shell interpretation, remote shell interpretation, awk interpretation or a literal string? I'm not sure how great an autoquoter is going to be if it doesn't know my intent. In the end, I'll need to do it myself, since I know what I want.

And the shame of that is that there aren't a lot of tools to help. Utility functions like shell_escape() would be nice, but you still need to handle escaping for the local shell. That's essentially true of any programming language.

Re: Zsh-autoquoter makes shell quoting slightly less annoying

#26

Post by haroldp »

Beltalowda wrote: post #18
haroldp wrote: post #8

Earlier quoted context omitted.

Just thinking of one of OP's examples, his (very cool) autoquoter allows him to type: > ssh user@host awk '{print $1}' file.txt And have the "$1" quoted or escaped so that it's interpreted as an awk keyword on the remote system. But what if I actually want it interpreted as a shell variable that should be expanded on the local or remote system? How does it know? Why is shell quoting still so hard?

> Why is shell quoting still so hard? The problem with the awk example here specifically is that you're mixing two different languages (shell and awk), and that's always going to be painful to a degree. Doing something like running Python code directly from a Ruby script also isn't fun, especially not if you want Python variables in that Ruby script. And to make it extra fun in the ssh you're also adding a second she…

> The problem with the awk example here specifically is that you're mixing two different languages (shell and awk)

Two languages, but three environments altogether. I'm running an awk script, in a remote shell script, in a local shell script. So a string with a meta-character or keyword within the awk script needs to be triply escaped. And there's no shell_escape(shell_escape(awk_escape(a literal dollar one $1))) available to make it less painful.

Re: Zsh-autoquoter makes shell quoting slightly less annoying

#27

Post by a1369209993 »

haroldp wrote: post #25
a1369209993 wrote: post #21

Earlier quoted context omitted.

> > ssh user@host awk '{print $1}' file.txt > it's interpreted as an awk keyword on the remote system. That's the behaviour in bash (and I think pretty much every other vaguely Bourne-like shell). That's what single quotes do . > what if I actually want it interpreted as a shell variable that should be expanded on the local [...] system? $ ssh user@host awk "{print $1}" file.txt But note that `$1` will be used as awk…

I mean, yes I know how to quote and escape. My point was that only I know what my intent was. Do I want local shell interpretation, remote shell interpretation, awk interpretation or a literal string? I'm not sure how great an autoquoter is going to be if it doesn't know my intent. In the end, I'll need to do it myself, since I know what I want. And the shame of that is that there aren't a lot of tools to help. Utili…

> My point was that only I know what my intent was. [...] In the end, I'll need to do it myself, since I know what I want.

I think we're in violent agreement here; my point was that the autoquoter is superfluous, since the command in your previous comment already does what

> his (very cool) autoquoter [supposedly] allows

even without a autoquoter.

> Utility functions like shell_escape() would be nice

Note that in this case you specifically do not want shell_escape(). You're trying to produce awk code that evaluates to a given string, and that requires knowing awk syntax, not shell syntax; if you escape according to shell syntax, a attacker may be able to find a string where the shell expression for that string, when interpreted as awk code, does something other than evalute to a string.

You could have a generic_escape() function that (say) replaced any non-alphanumeric byte with '\xHH' or '\B' (for 'B'==0xHH), but there will always some language where whatever generic strategy you picked doesn't work.

  $ echo "\x3E"
  x3E  # shell doesn't accept \xHH
  $ echo '12+3
Post reply on HN