Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A surprise with how '#!' handles its program argument in practice

utcc.utoronto.ca

Post reply on HN
Print view View original

41–50 of 116 posts

Re: A surprise with how '#!' handles its program argument in practice

#41

Post by opello »

tombert wrote: post #13
adastra22 wrote: post #8

Earlier quoted context omitted.

The kernel interprets the shebang line, not the shell.

I'm not sure the reason then, but they're definitely right; it works fine with zsh, doesn't work with bash. I wrote a test script to try it myself. I don't have fish installed and can't be bothered to go that far, but I suspect they're right about that as well.

It is strange, cursory digging for an explanation was a little more complex than I bargained for...

https://github.com/torvalds/linux/blob/v6.17/fs/binfmt_scrip...

I think it makes it to calling open_exec but there's a test for BINPRM_FLAGS_PATH_INACCESSIBLE, which doesn't seem relevant since 'bash' isn't like '/dev/fd//..', but does provoke an ENOENT.

https://github.com/torvalds/linux/blob/v6.17/fs/exec.c#L1445

Maybe someone else can explain it, I'd enjoy the details, and ran out of steam.

Re: A surprise with how '#!' handles its program argument in practice

#42

Post by userbinator »

I wonder what the reason was for having the kernel handle this, instead of the shell? To allow programs besides the shell to execute interpreted scripts as if they were actual binaries?

This is of course in stark contrast to dynamic linking, which is performed by a userspace program instead of the kernel, and much like the #!, this "interpreter"'s path is also hardcoded in dynamically linked binaries.

Re: A surprise with how '#!' handles its program argument in practice

#43

Post by theblazehen »

cm2187 wrote: post #35
rmunn wrote: post #31

Hit the link expecting to read about UTF-8 Byte Order Marks at the top of the file, so that the first few bytes aren't actually #! but 0xEF 0xBB 0xBF #! instead. Ran into this one just a few months ago when a coworker who uses Windows had checked a Bash script into the Git repo. His editor was configured to save files as "UTF-8 with BOM" and so we were getting errors that looked like "./doit.sh: line 1: #!/bin/bash:…

tbh it is lame for any program reading a text file to not support BOM. It's just one if.

There isn't really any one "text file" though, the kernel looks for the first two bytes to match what "#!" corresponds to in ASCII.

https://www.youtube.com/watch?v=J8nblo6BawU is some great watching on how "Plain text isn't that simple"

Re: A surprise with how '#!' handles its program argument in practice

#44

Post by coppsilgold »

There is no security issue here. The file with the '#!' needs to be executable, and at that point it doesn't matter what it invokes because you made it executable. It could have shellcode in it or it could call python3 which can also execute shellcode. Or more likely, it would just be a malware binary which you deliberately gave permissions to and executed.

Re: A surprise with how '#!' handles its program argument in practice

#45

Post by saurik »

jolmg wrote: post #27
saurik wrote: post #18

Earlier quoted context omitted.

Or, like, if you aren't reading and caring about what the interpreter is--as that's the only time this can burn you: it isn't doing a PATH lookup, so you can't walk into this one on accident--then it could literally be something like /bin/rm on some key file. This entire article is based on an assumption that this is somehow so obviously bad that there doesn't even need to be an explanation or defense of any kind of…

> This entire article is based on an assumption that this is somehow so obviously bad that there doesn't even need to be an explanation or defense of any kind of that idea. I'm not reading it like that. The tone is just one of surprise, since this isn't something that one typically sees. Since it's obscure, it leads one to wonder if it can be bad, and I don't see how it could be. I think it survived in the independen…

Right: I agree with you. I'm saying the article is making an unfounded assumption and am providing more reasoning for why you are correct.

Re: A surprise with how '#!' handles its program argument in practice

#46

Post by hulitu »

nflekkhnnn wrote: post #7
tombert wrote: post #2

Huh. I wish I had known this before. NixOS is annoying because everything is weird and symlinked and so I find myself fairly frequently making the mistake of writing `#!/bin/bash`, only to be told it can't find it, and I have to replace the path with `/run/current-system/sw/bin/bash`. Or at least I thought I did; apparently I can just have done `#!bash`. I just tested this, and it worked fine. You learn something new…

Anything other than ”#!/usr/bin/env bash” is doomed to fail at some time.

> Anything other than ”#!/usr/bin/env bash” is doomed to fail at some time.

if you have /usr/bin/env

Re: A surprise with how '#!' handles its program argument in practice

#47

Post by stabbles »

userbinator wrote: post #42

I wonder what the reason was for having the kernel handle this, instead of the shell? To allow programs besides the shell to execute interpreted scripts as if they were actual binaries? This is of course in stark contrast to dynamic linking, which is performed by a userspace program instead of the kernel, and much like the #!, this "interpreter"'s path is also hardcoded in dynamically linked binaries.

In a call `execve("/my/script", ...)` of course the kernel has to figure out how to run it, there is no shell involved.

As for scripts vs elf executables, there's not much of a difference between the shebang line and PT_INTERP, just that parsing shebangs lines is simpler.

Re: A surprise with how '#!' handles its program argument in practice

#48

Post by Arch-TK »

jolmg wrote: post #23
normie3000 wrote: post #20

Earlier quoted context omitted.

And is this shebang guaranteed to work always? Why isn't it more common?

Because /bin is the standard location for bash. The only one that breaks that expectation is NixOS (and maybe GuixSD?), apparently. I'm surprised they didn't symlink /bin or put a stub. Last time I tried NixOS was like 10 years ago. I thought there was a /bin/bash, but maybe it was just a /bin/sh? Other interpreters like python, ruby, etc. have more likelyhood of being used with "virtual environments", so it's more c…

/bin is the "standard" location for bash on a subset of Linux distributions and basically no other Unix...

So it's not really a standard.

/bin/sh is a much more common convention but once again, not a standard.

There really isn't a truly portable shebang, but the same can be said about executables themselves. As part of the build or install step of whatever thing you're making, you should really be looking these up and changing them.

What's more, bash isn't a standard shell.

Re: A surprise with how '#!' handles its program argument in practice

#49

Post by jolmg »

Arch-TK wrote: post #48
jolmg wrote: post #23

Earlier quoted context omitted.

Because /bin is the standard location for bash. The only one that breaks that expectation is NixOS (and maybe GuixSD?), apparently. I'm surprised they didn't symlink /bin or put a stub. Last time I tried NixOS was like 10 years ago. I thought there was a /bin/bash, but maybe it was just a /bin/sh? Other interpreters like python, ruby, etc. have more likelyhood of being used with "virtual environments", so it's more c…

/bin is the "standard" location for bash on a subset of Linux distributions and basically no other Unix... So it's not really a standard. /bin/sh is a much more common convention but once again, not a standard. There really isn't a truly portable shebang, but the same can be said about executables themselves. As part of the build or install step of whatever thing you're making, you should really be looking these up a…

Sorry, I should probably think more widely, but I was just considering Linux distros.

> /bin is the "standard" location for bash on a subset of Linux distributions

Considering "location" such that it includes /bin symlinks, that would be nearly all distros, I would think...

> What's more, bash isn't a standard shell.

De facto and specifically among Linux distros, it is. It's probably an underestimate that 95% of all Linux distro installations have it preinstalled.

Re: A surprise with how '#!' handles its program argument in practice

#50

Post by latexr »

Arch-TK wrote: post #48
jolmg wrote: post #23

Earlier quoted context omitted.

Because /bin is the standard location for bash. The only one that breaks that expectation is NixOS (and maybe GuixSD?), apparently. I'm surprised they didn't symlink /bin or put a stub. Last time I tried NixOS was like 10 years ago. I thought there was a /bin/bash, but maybe it was just a /bin/sh? Other interpreters like python, ruby, etc. have more likelyhood of being used with "virtual environments", so it's more c…

/bin is the "standard" location for bash on a subset of Linux distributions and basically no other Unix... So it's not really a standard. /bin/sh is a much more common convention but once again, not a standard. There really isn't a truly portable shebang, but the same can be said about executables themselves. As part of the build or install step of whatever thing you're making, you should really be looking these up a…

> /bin is the "standard" location for bash on a subset of Linux distributions and basically no other Unix...

You’re forgetting macOS. It has been using /bin/bash forever.

Post reply on HN