Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Path Isn't Real on Linux

blog.danielh.cc

Post reply on HN
Print view View original

81–90 of 120 posts

Re: Path Isn't Real on Linux

#81

Post by wahern »

opello wrote: post #50
matheusmoreira wrote: post #37

Earlier quoted context omitted.

On Linux the main boundary between user space and kernel is quite clear: the system call layer. It is stable and well documented. https://github.com/torvalds/linux/blob/master/Documentation/... System libraries like glibc are not part of the kernel, they are just components that can be replaced. I wrote an article about it: https://www.matheusmoreira.com/articles/linux-system-calls I even asked Greg Kroah-Hartman abo…

That is indeed one of the more well defined boundaries in the system. Also worth understanding is that programs aren't generally invoking system calls directly, for example calling interrupt 0x80, glibc provides wrapper functions that invoke system calls, blurring the boundary a bit. Further blurring the boundary is the vDSO layer that intercepts some system call wrappers for more efficient access. At issue in this a…

> vDSO layer that intercepts some system call wrappers for more efficient access.

Technically the vDSO library doesn't intercept. libc chooses to use either the vDSO or the syscall. This can happen either in the wrapper itself, or through a special PLT helper where the linker asks libc to resolve the symbol to populate the GOT entry. vDSO symbols have the prefix __kernel_ or __vdso_.

Re: Path Isn't Real on Linux

#82

Post by eternauta3k »

mzajc wrote: post #25

Fun fact: if you've ever had bash (or another shell) complain that a file doesn't exist, even though it's on $PATH, check if it's been cached by `hash`. If the file is moved elsewhere on $PATH and bash has the old path cached, you will get an ENOENT. The entire cache can be invalidated with `hash -r`.

The other typical cause is when an interpreter or library is compiled with the wrong libc version.

Re: Path Isn't Real on Linux

#83

Post by Hackbraten »

Aloisius wrote: post #49
vlovich123 wrote: post #38

Earlier quoted context omitted.

Is this an old behavior? I would think ENOENT would invalidate the cache entry at least.

It's still a thing in bash 5.2.37. It's just how bash works. If there's an entry in the session cache, it uses it. Since executable paths only get cached when you run a command successfully, this only happens when it gets moved from one directory in your PATH to another after you run it once, which isn't that common. Setting PATH or calling hash -r will clear the session cache or one could run set +h which will disab…

> this only happens when it gets moved from one directory in your PATH to another after you run it once

It also happens when you have two executables in different directories and then you delete the one with the higher priority. Happens regularly for me after I uninstall a Linux Homebrew package.

Re: Path Isn't Real on Linux

#84

Post by opello »

wahern wrote: post #81
opello wrote: post #50

Earlier quoted context omitted.

That is indeed one of the more well defined boundaries in the system. Also worth understanding is that programs aren't generally invoking system calls directly, for example calling interrupt 0x80, glibc provides wrapper functions that invoke system calls, blurring the boundary a bit. Further blurring the boundary is the vDSO layer that intercepts some system call wrappers for more efficient access. At issue in this a…

> vDSO layer that intercepts some system call wrappers for more efficient access. Technically the vDSO library doesn't intercept. libc chooses to use either the vDSO or the syscall. This can happen either in the wrapper itself, or through a special PLT helper where the linker asks libc to resolve the symbol to populate the GOT entry. vDSO symbols have the prefix __kernel_ or __vdso_.

That's fair, sorry for my casual language in a technically nuanced discussion. I hadn't looked at this in quite a while, but it was good to review. Thanks for the prompting.

https://github.com/bminor/glibc/blob/glibc-2.41/sysdeps/unix...

https://github.com/bminor/glibc/blob/glibc-2.41/sysdeps/unix...

Re: Path Isn't Real on Linux

#85

Post by teo_zero »

AlienRobot wrote: post #52
amiga386 wrote: post #41

Earlier quoted context omitted.

* in Linux, the kernel is responsible for accepting a list of execve(2) argument-words Yes it does, but the more surprising thing is (coming from AmigaOS with its dos.library function ReadArgs()) that the shell does this . The shell is also responsible for argument expansion - madness! On AmigaOS, when you type "delete foo#? force", the shell passes the entire command line to the delete command. The delete command ca…

Username checks out. I think one of the most surprising things I learned about bash is that you can do this: touch ./-rf rm * And now you have rm -rf'd. :)

We should use "--" more, but who has all this time to waste? :)

Re: Path Isn't Real on Linux

#87

Post by wahern »

opello wrote: post #84
wahern wrote: post #81

Earlier quoted context omitted.

> vDSO layer that intercepts some system call wrappers for more efficient access. Technically the vDSO library doesn't intercept. libc chooses to use either the vDSO or the syscall. This can happen either in the wrapper itself, or through a special PLT helper where the linker asks libc to resolve the symbol to populate the GOT entry. vDSO symbols have the prefix __kernel_ or __vdso_.

That's fair, sorry for my casual language in a technically nuanced discussion. I hadn't looked at this in quite a while, but it was good to review. Thanks for the prompting. https://github.com/bminor/glibc/blob/glibc-2.41/sysdeps/unix... https://github.com/bminor/glibc/blob/glibc-2.41/sysdeps/unix...

I also double-checked the glibc and musl code to make sure I wasn't misremembering, and ended up learning about IFUNC.[1] Previously I had avoided going down the rabbit hole to understand what glibc's libc_ifunc was doing. I don't think musl uses IFUNC, at least not for clock_gettime; it seems to always link the wrapper which calls the vdso through an internally managed pointer.[2]

And now I'm wondering how safe all this indirection is. For the PLT/GOT approach I think you can disable lazy binding and force the GOT table to be read-only so exploits can't overwrite the symbol addresses. But for musl's approach it doesn't seem like you can make it's internal function pointer read-only, though maybe it's more difficult to find the address of than GOT table slots.

[1] https://sourceware.org/glibc/wiki/GNU_IFUNC [2] https://git.musl-libc.org/cgit/musl/tree/src/time/clock_gett...

Re: Path Isn't Real on Linux

#88

Post by Joker_vD »

Aloisius wrote: post #70
Joker_vD wrote: post #55

Earlier quoted context omitted.

Which is the universally known convention everyone is born with inherent knowledge of. Also, people read man-pages.

What person diving into their shell's source code on Linux doesn't read manpages? Or even man's manpage at least once?

Daniel Huang, the one that wrote TFA? People are different, I don't know what else to tell you. But generally, people don't read man pages or docs.

Re: Path Isn't Real on Linux

#89

Post by Joker_vD »

matheusmoreira wrote: post #64
opello wrote: post #50

Earlier quoted context omitted.

That is indeed one of the more well defined boundaries in the system. Also worth understanding is that programs aren't generally invoking system calls directly, for example calling interrupt 0x80, glibc provides wrapper functions that invoke system calls, blurring the boundary a bit. Further blurring the boundary is the vDSO layer that intercepts some system call wrappers for more efficient access. At issue in this a…

> Also worth understanding is that programs aren't generally invoking system calls directly They don't generally do that but they absolutely can. I wrote a Lisp interpreter that does just that. It's completely static, has zero dependencies and talks to the kernel directly. The idea is to implement every primitive on top of Linux, and everything else on top of the primitives. From the kernel's perspective, every progr…

> It took years for glibc to provide getrandom.

A tangent: Robert Clausecker, the guy who submitted the proposal for adding tcgetwinsize() and SIGWINCH to POSIX, apparently did it because it "is probably the easiest way to get glibc to implement a feature you want" [0].

[0] https://news.ycombinator.com/item?id=42041467

Re: Path Isn't Real on Linux

#90

Post by Joker_vD »

smcameron wrote: post #79

> The shell, not the Linux kernel, is responsible for searching for executables in PATH! I mean, no shit, Sherlock? the exec family of system calls requires a path to a file, not a filename with an implicit path from the environment, of course the PATH is handled by the shell.

All members of the exec family of system calls, which consists of only two syscalls, namely, execve(2) and execveat(2), literally have the envp parameter which is supposed to have all the environment variables for the process.

Now, the semantics of this parameter is that kernel does not use it for path resolution when searching for the executable — but it could.

Post reply on HN