Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Not knowing the /proc file system

admccartney.mur.at

Post reply on HN
Print view View original

51–60 of 111 posts

Re: Not knowing the /proc file system

#51

Post by twic »

Galanwe wrote: post #8
Animats wrote: post #6

The Linux /proc "file system" is kernel to user space communication hammered into the wrong form because "everything is a file". /proc is a system call with a fake file system API, and this matters. The sample code won't work reliably, because it assumes that the "files" won't change while being read. If you read /proc, you must "read" each file with one unbuffered kernel read to be free of race conditions. See [1].…

I generally agree with your comment, especially coming from OpenBSD where /proc does not exist, for reasons you mentioned, as well as a purely practical one: let's not pressure the VFS for no reason. I think the biggest issue with /proc at the moment is that it leaks in core Linux components, and getting rid of it is becoming impossible. Some years ago I worked on adding `$ORIGIN` rpath support in OpenBSD and looked…

> One thing though, you focus a lot on syscalls, but I think the challenge in recent years has been to find new ways to have kernel userspace interactions _outside_ of syscall, which are cumbersome to use, rigid in structure, and practically speaking, there are only so many entries you can store in an IDT.

There's always ioctls. You could have something a lot like /proc and /sys, with a directory hierarchy modelling various interesting things, but rather than the files having textual contents, you interact with them via ioctls. ioctls are identified by a device-specific integer, so you have masses of namespace (about 2*32 operations per device type, and you can have a lot of device types). They take or receive a block of memory in one go, so there is no parsing or formatting or worrying about inconsistent reads. I think this is even simpler than protocol-esque interfaces like netlink.

Re: Not knowing the /proc file system

#52

Post by [deleted] »

twic wrote: post #51
Galanwe wrote: post #8

Earlier quoted context omitted.

I generally agree with your comment, especially coming from OpenBSD where /proc does not exist, for reasons you mentioned, as well as a purely practical one: let's not pressure the VFS for no reason. I think the biggest issue with /proc at the moment is that it leaks in core Linux components, and getting rid of it is becoming impossible. Some years ago I worked on adding `$ORIGIN` rpath support in OpenBSD and looked…

> One thing though, you focus a lot on syscalls, but I think the challenge in recent years has been to find new ways to have kernel userspace interactions _outside_ of syscall, which are cumbersome to use, rigid in structure, and practically speaking, there are only so many entries you can store in an IDT. There's always ioctls. You could have something a lot like /proc and /sys, with a directory hierarchy modelling…

[deleted]

Re: Not knowing the /proc file system

#53

Post by JdeBP »

chasil wrote: post #50
Animats wrote: post #6

The Linux /proc "file system" is kernel to user space communication hammered into the wrong form because "everything is a file". /proc is a system call with a fake file system API, and this matters. The sample code won't work reliably, because it assumes that the "files" won't change while being read. If you read /proc, you must "read" each file with one unbuffered kernel read to be free of race conditions. See [1].…

I believe that the /proc filesystem first appeared commercially in Solaris, and was then adopted and expanded in Linux. I got an SDF account recently, and was surprised to find it in NetBSD. OpenBSD has great resistance to it. ...looking at the wiki, many more kernels implement /proc: "Many Unix-like operating systems support the proc filesystem, including Solaris, IRIX, Tru64 UNIX, BSD, Linux, IBM AIX, QNX, and Plan…

The problem is Wikipedia being misleading again. They each support a proc filesystem. There is no single the proc filesystem that they support.

A case in point: FreeBSD's /proc is very different to Linux's /proc, and most of what one would go to /proc on Linux for is obtained via sysctl() on FreeBSD, with a lot less in the way of machine readable → human readable → machine readable busywork formatting and re-parsing involved.

Re: Not knowing the /proc file system

#54

Post by chasil »

Here is a little shell script to print out what is easily seen in /proc/*/cmdline.

This requires a GNU xargs that supports NULL termination (or something compatible); as I understand it, this cannot be done with POSIX tools.

  $ cat shps
  #!/bin/dash

  for path in /proc/*/cmdline
  do p=${path#*/} p=${p#*/} p=${p%/*}
     case "$p" in *[!0-9]*) continue;; esac
     c="$(xargs -0 echo

Re: Not knowing the /proc file system

#55

Post by [deleted] »

Someone wrote: post #47
dezgeg wrote: post #37

Earlier quoted context omitted.

In practice signals don't seem to be a problem for (most?) proc files as the seq_file infrastructure doesn't bother to check for pending signals and terminate the read if so. Naturally, only source reading will answer whether the proc file you're interested in is vulnerable to this or not.

> Naturally, only source reading will answer whether the proc file you're interested in is vulnerable to this or not. At the moment. To be sure, you also have to keep track of changes to that source code to look for regressions, and have a mechanism to update all your deployed code if there is a regression there. It could require multiple attempts at reading, and still doesn’t ensure progress, but I think /proc would…

[deleted]

Re: Not knowing the /proc file system

#56

Post by Galanwe »

twic wrote: post #51
Galanwe wrote: post #8

Earlier quoted context omitted.

I generally agree with your comment, especially coming from OpenBSD where /proc does not exist, for reasons you mentioned, as well as a purely practical one: let's not pressure the VFS for no reason. I think the biggest issue with /proc at the moment is that it leaks in core Linux components, and getting rid of it is becoming impossible. Some years ago I worked on adding `$ORIGIN` rpath support in OpenBSD and looked…

> One thing though, you focus a lot on syscalls, but I think the challenge in recent years has been to find new ways to have kernel userspace interactions _outside_ of syscall, which are cumbersome to use, rigid in structure, and practically speaking, there are only so many entries you can store in an IDT. There's always ioctls. You could have something a lot like /proc and /sys, with a directory hierarchy modelling…

Agree, ioctls are a good replacement of /proc for exposing simple, structured information to/from the kernel.

It is still not perfect for all use cases though, especially because it forces you to operate in a _polling_ fashion.

Programs like top/iotop/htop are a good example (also, who is not guilty of abusing one liners a-la `watch -n1 cat /proc/meminfo`?). Instead of polling /proc or ioctls, you would really like to register on events, and get notifications pushed.

Netlink is well suited for these kinds of scenarios, though mostly to get notified of network interface changes last I checked.

Re: Not knowing the /proc file system

#57

Post by io_urine »

Galanwe wrote: post #8
Animats wrote: post #6

The Linux /proc "file system" is kernel to user space communication hammered into the wrong form because "everything is a file". /proc is a system call with a fake file system API, and this matters. The sample code won't work reliably, because it assumes that the "files" won't change while being read. If you read /proc, you must "read" each file with one unbuffered kernel read to be free of race conditions. See [1].…

I generally agree with your comment, especially coming from OpenBSD where /proc does not exist, for reasons you mentioned, as well as a purely practical one: let's not pressure the VFS for no reason. I think the biggest issue with /proc at the moment is that it leaks in core Linux components, and getting rid of it is becoming impossible. Some years ago I worked on adding `$ORIGIN` rpath support in OpenBSD and looked…

> and practically speaking, there are only so many entries you can store in an IDT.

The number of syscalls isn't limited by this, they all go through the same interrupt and are dispatched to a specific handler function based on the value of a register.

For example this is how it works on arm64: https://elixir.bootlin.com/linux/v6.5/source/arch/arm64/kern...

And on x86: https://elixir.bootlin.com/linux/v6.5/source/arch/x86/entry/...

Re: Not knowing the /proc file system

#58

Post by Galanwe »

o11c wrote: post #48
Galanwe wrote: post #8

Earlier quoted context omitted.

I generally agree with your comment, especially coming from OpenBSD where /proc does not exist, for reasons you mentioned, as well as a purely practical one: let's not pressure the VFS for no reason. I think the biggest issue with /proc at the moment is that it leaks in core Linux components, and getting rid of it is becoming impossible. Some years ago I worked on adding `$ORIGIN` rpath support in OpenBSD and looked…

> Some years ago I worked on adding `$ORIGIN` rpath support in OpenBSD and looked at how Linux did it. Mind you, the dynamic linker (ldlinux.so) grabs the running executable path from /proc... `getauxval(AT_EXECFN)` is "better" but there are weird edge cases with `fexecve` where you only get `AT_EXECFD`, even ignoring the inevitable TOCTOU.

I am a bit blurry on the details because I worked on that almost 10 years ago to support a program that used (and abused) `$ORIGIN`.

I ended up pushing the executable filename through the aux vector to the dynamic linker, which is simar to what you mention AFAIU.

Never managed to get that merged upstream though. Glad to know there's now a _standard_ way.

Re: Not knowing the /proc file system

#60

Post by twic »

Galanwe wrote: post #56
twic wrote: post #51

Earlier quoted context omitted.

> One thing though, you focus a lot on syscalls, but I think the challenge in recent years has been to find new ways to have kernel userspace interactions _outside_ of syscall, which are cumbersome to use, rigid in structure, and practically speaking, there are only so many entries you can store in an IDT. There's always ioctls. You could have something a lot like /proc and /sys, with a directory hierarchy modelling…

Agree, ioctls are a good replacement of /proc for exposing simple, structured information to/from the kernel. It is still not perfect for all use cases though, especially because it forces you to operate in a _polling_ fashion. Programs like top/iotop/htop are a good example (also, who is not guilty of abusing one liners a-la `watch -n1 cat /proc/meminfo`?). Instead of polling /proc or ioctls, you would really like t…

That's an interesting point. I wonder if some sort of polling interface to ioctls would be feasible. On top of kqueue or io_uring or something maybe.
Post reply on HN