Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A surprise with how '#!' handles its program argument in practice

utcc.utoronto.ca

Post reply on HN
Print view View original

31–40 of 116 posts

Re: A surprise with how '#!' handles its program argument in practice

#31

Post by rmunn »

Hit the link expecting to read about UTF-8 Byte Order Marks at the top of the file, so that the first few bytes aren't actually #! but 0xEF 0xBB 0xBF #! instead. Ran into this one just a few months ago when a coworker who uses Windows had checked a Bash script into the Git repo. His editor was configured to save files as "UTF-8 with BOM" and so we were getting errors that looked like "./doit.sh: line 1: #!/bin/bash: No such file or directory". Can you see the invisible BOMb in that line? It's there, I promise you.

That's not what the article was actually about, as it turned out. The surprise in the article was about relative paths for script shebang lines. Which was useful to learn about, of course, but I was actually surprised by the surprise.

Re: A surprise with how '#!' handles its program argument in practice

#32

Post by jolmg »

somat wrote: post #29

The thing that surprised me was that you can't write an interpreter in an interpreted language, at least not in obsd. It is possible if you jump through a few hoops but you can't directly call it. An example: if you made a language in python /bin/my_lang: #does nothing but pretend it does #!/usr/local/bin/python3 import sys print('my_lang args', sys.argv) for line in sys.stdin: print('invalid_line:', line) my_script:…

Worked for me, but the way you described it has issues:

1. You chmod my_script twice.

2. Did you chmod u+x /bin/my_lang too? Since you put it in /bin, are you sure the owner isn't root?, in which case your user wouldn't have execute permission. Try +x instead of u+x.

3. Do you have python in that path? Try `/usr/bin/env python` instead.

4. In case you expected otherwise, my_script wouldn't be passed through stdin. It's just provided as an argument to my_lang.

Re: A surprise with how '#!' handles its program argument in practice

#33

Post by [deleted] »

rmunn wrote: post #31

Hit the link expecting to read about UTF-8 Byte Order Marks at the top of the file, so that the first few bytes aren't actually #! but 0xEF 0xBB 0xBF #! instead. Ran into this one just a few months ago when a coworker who uses Windows had checked a Bash script into the Git repo. His editor was configured to save files as "UTF-8 with BOM" and so we were getting errors that looked like "./doit.sh: line 1: #!/bin/bash:…

[deleted]

Re: A surprise with how '#!' handles its program argument in practice

#34

Post by opello »

jolmg wrote: post #11
adastra22 wrote: post #8

Earlier quoted context omitted.

The kernel interprets the shebang line, not the shell.

It is possible for the shell to handle it. From zshall(1): > If the program is a file beginning with ‘#!', the remainder of the first line specifies an interpreter for the program. The shell will execute the specified interpreter on operating systems that do not handle this executable format in the kernel. Taking a quick look at the source in Src/exec.c: execve(pth, argv, newenvp); // [...] if ((eno = errno) == ENOEX…

I did a little digging and found that the `|| eno == ENOENT` was added quite a bit earlier[1] than the actual pathprog lookup[2]. While I could find the "issue discussion" for the pathprog change[3] I wasn't able to find it for the ENOENT addition, which was kind of interesting and frustrating--[4] is the `X-Seq` mentioned in the commit but that seems to be inconsistent or incorrect for the actual cross-reference, and nearby in time wasn't helpful either.

[1] https://sourceforge.net/p/zsh/code/ci/29ed6c7e3ab32da20f528a...

[2] https://sourceforge.net/p/zsh/code/ci/29ed6c7e3ab32da20f528a...

[3] https://www.zsh.org/mla/workers/2010/msg00522.html

[4] https://www.zsh.org/mla/workers/2000/msg01168.html

Re: A surprise with how '#!' handles its program argument in practice

#35

Post by cm2187 »

rmunn wrote: post #31

Hit the link expecting to read about UTF-8 Byte Order Marks at the top of the file, so that the first few bytes aren't actually #! but 0xEF 0xBB 0xBF #! instead. Ran into this one just a few months ago when a coworker who uses Windows had checked a Bash script into the Git repo. His editor was configured to save files as "UTF-8 with BOM" and so we were getting errors that looked like "./doit.sh: line 1: #!/bin/bash:…

tbh it is lame for any program reading a text file to not support BOM. It's just one if.

Re: A surprise with how '#!' handles its program argument in practice

#36

Post by teo_zero »

somat wrote: post #29

The thing that surprised me was that you can't write an interpreter in an interpreted language, at least not in obsd. It is possible if you jump through a few hoops but you can't directly call it. An example: if you made a language in python /bin/my_lang: #does nothing but pretend it does #!/usr/local/bin/python3 import sys print('my_lang args', sys.argv) for line in sys.stdin: print('invalid_line:', line) my_script:…

Wait... your interpreter reads from stdin. Shouldn't it read its first arg, instead?

Re: A surprise with how '#!' handles its program argument in practice

#37

Post by somat »

jolmg wrote: post #32
somat wrote: post #29

The thing that surprised me was that you can't write an interpreter in an interpreted language, at least not in obsd. It is possible if you jump through a few hoops but you can't directly call it. An example: if you made a language in python /bin/my_lang: #does nothing but pretend it does #!/usr/local/bin/python3 import sys print('my_lang args', sys.argv) for line in sys.stdin: print('invalid_line:', line) my_script:…

Worked for me, but the way you described it has issues: 1. You chmod my_script twice. 2. Did you chmod u+x /bin/my_lang too? Since you put it in /bin, are you sure the owner isn't root?, in which case your user wouldn't have execute permission. Try +x instead of u+x. 3. Do you have python in that path? Try `/usr/bin/env python` instead. 4. In case you expected otherwise, my_script wouldn't be passed through stdin. It…

I am on openbsd. which does not allow it, it looks like nested interpreters are s supported on linux. So my loss there.

Re: A surprise with how '#!' handles its program argument in practice

#38

Post by [deleted] »

cm2187 wrote: post #35
rmunn wrote: post #31

Hit the link expecting to read about UTF-8 Byte Order Marks at the top of the file, so that the first few bytes aren't actually #! but 0xEF 0xBB 0xBF #! instead. Ran into this one just a few months ago when a coworker who uses Windows had checked a Bash script into the Git repo. His editor was configured to save files as "UTF-8 with BOM" and so we were getting errors that looked like "./doit.sh: line 1: #!/bin/bash:…

tbh it is lame for any program reading a text file to not support BOM. It's just one if.

[deleted]

Re: A surprise with how '#!' handles its program argument in practice

#39

Post by somat »

teo_zero wrote: post #36
somat wrote: post #29

The thing that surprised me was that you can't write an interpreter in an interpreted language, at least not in obsd. It is possible if you jump through a few hoops but you can't directly call it. An example: if you made a language in python /bin/my_lang: #does nothing but pretend it does #!/usr/local/bin/python3 import sys print('my_lang args', sys.argv) for line in sys.stdin: print('invalid_line:', line) my_script:…

Wait... your interpreter reads from stdin. Shouldn't it read its first arg, instead?

I think that was what I was trying to figure out, how the program was passed. but OpenBSD does not do nested interpreters, it looks like if I had tried Linux it would have worked.

Re: A surprise with how '#!' handles its program argument in practice

#40

Post by somat »

jolmg wrote: post #23
normie3000 wrote: post #20

Earlier quoted context omitted.

And is this shebang guaranteed to work always? Why isn't it more common?

Because /bin is the standard location for bash. The only one that breaks that expectation is NixOS (and maybe GuixSD?), apparently. I'm surprised they didn't symlink /bin or put a stub. Last time I tried NixOS was like 10 years ago. I thought there was a /bin/bash, but maybe it was just a /bin/sh? Other interpreters like python, ruby, etc. have more likelyhood of being used with "virtual environments", so it's more c…

Will not wok on OpenBSD where the shell that comes with the system is ksh at /bin/ksh and /bin/sh and if you want bash it is a third party package and correspondingly gets installed as /usr/local/bin/bash

It does get awkward, especially when porting. all your third party libraries and includes are in /usr/local/lib /usr/local/include but at least it is better than freebsd which also insists on putting all third party configs under /usr/local/etc/

Post reply on HN