Live data from Hacker News

A surprise with how '#!' handles its program argument in practice

utcc.utoronto.ca

81–90 of 116 posts

Re: A surprise with how '#!' handles its program argument in practice

#81

The authors previous article about "...(not) using #!/usr/bin/env whatever" doesn't sit well with me. "The only reason to start your script with '#!/usr/bin/env ' is if you expect your script to run on a system where Bash or whatever else isn't where you expect (or when it has to run on systems that have ' ' in different places, which is probably most common for third party packages)." His very first point is how you…

NixOS only has /usr/bin/env and /bin/sh.

Re: A surprise with how '#!' handles its program argument in practice

#82
post #77

The authors previous article about "...(not) using #!/usr/bin/env whatever" doesn't sit well with me. "The only reason to start your script with '#!/usr/bin/env ' is if you expect your script to run on a system where Bash or whatever else isn't where you expect (or when it has to run on systems that have ' ' in different places, which is probably most common for third party packages)." His very first point is how you…

Never thought of it this way; isn’t it always safe to assume env is in PATH? Maybe `#! env ` could be considered a DSL for hashbangs. My reasoning is that `/usr/bin/env` is the thing that seems to be hard-coded to a system path, in most cases.

[deleted]

Re: A surprise with how '#!' handles its program argument in practice

#83
post #78
post #54

Earlier quoted context omitted.

/usr/bin/env and /bin/sh are part of the POSIX standard, this is why NixOS has those available.

> /usr/bin/env and /bin/sh are part of the POSIX standard, this is why NixOS has those available. Contrary to popular belief, those aren't in the POSIX standard. The following are not in the POSIX standard, they are just widely implemented: - "#!" line. - /bin/sh as the location of a POSIX compliant shell, or any shell. - /usr/bin/env as the location of an env program. - The -S option to env.

I think you are using "not required by the POSIX standard" when you say "not in" which is not an accurate shorthand.

#! is certainly in the POSIX standard as the exact topic of "is /bin/sh always a POSIX" shell is a discussion point (it is not guaranteed since there were systems that existed at the time that had a non-POSIX shell there)

Re: A surprise with how '#!' handles its program argument in practice

#84

The authors previous article about "...(not) using #!/usr/bin/env whatever" doesn't sit well with me. "The only reason to start your script with '#!/usr/bin/env ' is if you expect your script to run on a system where Bash or whatever else isn't where you expect (or when it has to run on systems that have ' ' in different places, which is probably most common for third party packages)." His very first point is how you…

I use `#!/usr/bin/env bash` because the /bin/bash on macOS is usually stupidly ancient, and I install a recent version of bash to ~/.local/bin/bash

Is it bad? Well it's less secure. But if you're worried about `/usr/bin/env` calling a malicious program then you need to call out the path for every executable in the script and there's a hell of a lot more other things to worry about too.

It's the same for `#!/usr/bin/env python3` in python scripts. Python3 itself might be ancient at system install, but you might need to be using a venv. So /usr/bin/env python3 works correctly while /usr/bin/python3 works incorrectly.

So is it bad? No.

Re: A surprise with how '#!' handles its program argument in practice

#85
post #75
post #66

I do this all the time for python virtualenvs. Instead of calling ‘#!/usr/bin/env python3’ I will call ‘#!venv/bin/python3’. This way I don’t have to worry about whether the environment was activated or not.

If you're okay with running the script only from the parent directory of the venv (I guess you have things set up so that's the project root), fine. You never have to "worry about whether the environment was activated", unless your code depends on those environment variables (in which case your shebang trick won't help you). Just specify the path to the venv's Python executable. You aren't really intended to put your…

I use venv's to keep library installs out of the global space. My python scripts won't run without those libraries, so running from the project root is pretty much required. I don't use the variables set by activating the venv, so that's not a real concern.

I've done this for years to keep my individual projects separate and not changing activations when switching directories. I also make sure to only call `venv/bin/pip` or `venv/bin/python3` when I'm directly calling pip or python. So, yes -- you have to be in the root project directory for this to work, but for me, that's a useful tradeoff. Even when running code from within a docker container, I still use this method, so I make sure that I'm executing from the proper work directory.

If I think that I need to run a program (without arguments), I'll have a short shell wrapper that is essentially:

    #!/bin/bash
    cd $(dirname $0)
    venv/bin/python3 myscript.py
As far as running a program that's managed by venv/pip, symlinks are essentially what I do. I'll create a new venv for each program, install it in that venv, and then symlink from venv/bin/program to $HOME/.local/bin/. It works very well when you're installing a pip managed program.

Re: A surprise with how '#!' handles its program argument in practice

#86
post #77

The authors previous article about "...(not) using #!/usr/bin/env whatever" doesn't sit well with me. "The only reason to start your script with '#!/usr/bin/env ' is if you expect your script to run on a system where Bash or whatever else isn't where you expect (or when it has to run on systems that have ' ' in different places, which is probably most common for third party packages)." His very first point is how you…

Never thought of it this way; isn’t it always safe to assume env is in PATH? Maybe `#! env ` could be considered a DSL for hashbangs. My reasoning is that `/usr/bin/env` is the thing that seems to be hard-coded to a system path, in most cases.

I think POSIX requires env but doesn't mandate a location for it.

Re: A surprise with how '#!' handles its program argument in practice

#87
post #52
post #46

Earlier quoted context omitted.

> Anything other than ”#!/usr/bin/env bash” is doomed to fail at some time. if you have /usr/bin/env

/usr/bin/env (and /bin/sh) are part of POSIX, that is why the above shebang is the recommended way to start a shell.

/bin/sh is NOT required by POSIX, they explicitly warn that it may not exist[1].

> Applications should note that the standard PATH to the shell cannot be assumed to be either /bin/sh or /usr/bin/sh, and should be determined by interrogation of the PATH returned by getconf PATH , ensuring that the returned pathname is an absolute pathname and not a shell built-in.

[1] https://pubs.opengroup.org/onlinepubs/9799919799/

Re: A surprise with how '#!' handles its program argument in practice

#88
post #13

Earlier quoted context omitted.

I'm not sure the reason then, but they're definitely right; it works fine with zsh, doesn't work with bash. I wrote a test script to try it myself. I don't have fish installed and can't be bothered to go that far, but I suspect they're right about that as well.

env bash is all well and good for normies, but if you're already on NixOS did you know you can have nix-shell be your interpreter and back flip into any reproducible interpreted environment you like? https://nixos.wiki/wiki/Nix-shell_shebang

Or any other system with Nix installed. I use this at work to provide scripts with all their dependencies specified that work across any Linux distro & MacOS. First execution is slow since it has to fetch everything, but after that it's fast and just works.

Re: A surprise with how '#!' handles its program argument in practice

#89
post #31

Hit the link expecting to read about UTF-8 Byte Order Marks at the top of the file, so that the first few bytes aren't actually #! but 0xEF 0xBB 0xBF #! instead. Ran into this one just a few months ago when a coworker who uses Windows had checked a Bash script into the Git repo. His editor was configured to save files as "UTF-8 with BOM" and so we were getting errors that looked like "./doit.sh: line 1: #!/bin/bash:…

UTF-8 doesn't have a BOM. UTF-16 does. UTF-32 does. "UTF-8 with BOM" is not a standards-compliant text format, it's a proprietary binary format that happens to have a bunch of embedded UTF-8. Just because you can run `strings` on a file & get a bunch of text out doesn't mean it's a text file!

Re: A surprise with how '#!' handles its program argument in practice

#90
post #35
post #31

Hit the link expecting to read about UTF-8 Byte Order Marks at the top of the file, so that the first few bytes aren't actually #! but 0xEF 0xBB 0xBF #! instead. Ran into this one just a few months ago when a coworker who uses Windows had checked a Bash script into the Git repo. His editor was configured to save files as "UTF-8 with BOM" and so we were getting errors that looked like "./doit.sh: line 1: #!/bin/bash:…

tbh it is lame for any program reading a text file to not support BOM. It's just one if.

UTF-8 is a text format with no BOM. Just like ASCII doesn't support a BOM. The BOM is a UTF-16 or UTF-32 thing, so "UTF-8 with BOM" is a binary file that happens to contain some UTF-8 strings as well. Since it's not a text file, it makes sense that utilities expecting text files don't handle it.
Post reply on HN