Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Not knowing the /proc file system

admccartney.mur.at

Post reply on HN
Print view View original

61–70 of 111 posts

Re: Not knowing the /proc file system

#61

Post by tczMUFlmoNk »

chasil wrote: post #54

Here is a little shell script to print out what is easily seen in /proc/*/cmdline. This requires a GNU xargs that supports NULL termination (or something compatible); as I understand it, this cannot be done with POSIX tools. $ cat shps #!/bin/dash for path in /proc/*/cmdline do p=${path#*/} p=${p#*/} p=${p%/*} case "$p" in *[!0-9]*) continue;; esac c="$(xargs -0 echo

> This requires a GNU xargs that supports NULL termination (or something compatible); as I understand it, this cannot be done with POSIX tools.

You can do this with `tr`, right?

    #!/bin/dash
    for path in /proc/*/cmdline
    do
        p=${path#*/} p=${p#*/} p=${p%/*}
        case "$p" in *[!0-9]*) continue;; esac
        c="$(tr '\0' ' '
Those escape sequences are covered in the standard: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/t...

(But please correct me if I'm missing something!)

This also avoids (POSIX-)undefined behavior with `echo` in case one of the processes' `argv[0]` happens to begin with `-n` or any argument contains a backslash.

Re: Not knowing the /proc file system

#62

Post by School-Cotton »

xyzelement wrote: post #28
rascul wrote: post #24

Earlier quoted context omitted.

> anOverlyLongAndInformationFreeIdentifier Apple has one 82 characters. https://developer.apple.com/documentation/contacts/cnlabelco...

And I would say that’s a good case where the long name is meaningful and necessary. On the other hand when you are dealing with a file pointer in a language that deals with file pointers constantly, “fp” is meaningful enough.

CNLabelContactRelationBiaoMei would have been a lot better. People can look up what that means when they need to.

Re: Not knowing the /proc file system

#63

Post by crabbone »

Galanwe wrote: post #8
Animats wrote: post #6

The Linux /proc "file system" is kernel to user space communication hammered into the wrong form because "everything is a file". /proc is a system call with a fake file system API, and this matters. The sample code won't work reliably, because it assumes that the "files" won't change while being read. If you read /proc, you must "read" each file with one unbuffered kernel read to be free of race conditions. See [1].…

I generally agree with your comment, especially coming from OpenBSD where /proc does not exist, for reasons you mentioned, as well as a purely practical one: let's not pressure the VFS for no reason. I think the biggest issue with /proc at the moment is that it leaks in core Linux components, and getting rid of it is becoming impossible. Some years ago I worked on adding `$ORIGIN` rpath support in OpenBSD and looked…

What about stuff like udev? Or anything that has a kernel module and a controlling program running in userspace, eg. iscisiadm, mdadm etc? This way any given utility is free to implement and experiment with the communication protocol however they want.

But, more generally, if we want general-purpose communication, why not press further and have relational database-like interface, with transactional semantics, triggers, fine-grained ownership?..

Re: Not knowing the /proc file system

#64

Post by enriquto »

9dev wrote: post #2

Tangent: Looking at those code samples, I wonder whether coming up with the shortest possible, most cryptic variable names is somewhat of a sport amongst C developers. Are you guys still coding in Notepad and need to conserve keystrokes, or where does that reluctance to use proper names come from?

> Tangent: Looking at those code samples, I wonder whether coming up with the shortest possible, most cryptic variable names is somewhat of a sport amongst C developers.

I definitely make a point of using only single-letter variables in most of my C and python programs.

It is a very common usage in scientific computing. In math, all variables are single letters. Always. If a variable has more than one letter, you read it as the product of several variables, one for each constituent letter. When you are translating a formula that reads "y=Ax", you want to write something like

    y = A * x
Writing this formula as

    output = operator * input
or, god forbid, as something like

    output = linalg.dot(operator, input)
is completely ridiculous to any mathematician.

Mathematics itself used to be like that in ancient times. But after centuries of distillation, we arrived to the modern efficient notation. Some "programmers" want us to go back to the ancient ways, writing simple formulas as full-sized English sentences. But they will take single-letter variable names from our cold, dead hands!

Of course, the first appearance of each single-letter variable must be accompanied by a comment describing what it is. But after this comment, you can use that letter as many times as you want. Encoding that information in the variable name itself would be disturbingly redundant if you use the variable more than once (which will be always the case).

Re: Not knowing the /proc file system

#65

Post by deathanatos »

> One thing that stands out is the call to ioctl - I have no idea why this is happening and it also appears to be causing an error. As far as I understand, the ioctl call signifies that the program is trying to do a terminal control operation. Dunno.

It's not an error, per se. (The ioctl is literally erroring, but that's an expected possibility for the calling code, and it handles that.)

The reason there's an ioctl is documented in the docs for `open`:

> buffering is an optional integer used to set the buffering policy. Pass 0 to switch buffering off (only allowed in binary mode), 1 to select line buffering (only usable in text mode), and an integer > 1 to indicate the size in bytes of a fixed-size chunk buffer.

You're not passing the `buffering` arg, so the subsequent text applies:

> When no buffering argument is given, the default buffering policy works as follows:

> * Binary files are buffered in fixed-size chunks; […]

(That doesn't apply, as you're not opening the file in binary mode, so it's the next bullet that applies)

> * Interactive” text files (files for which isatty() returns True) use line buffering. Other text files use the policy described above for binary files.

That ioctl is the underlying syscall that isatty() is calling. It's determining if the opened file is a TTY, or not. The file isn't a TTY, so the ioctl returns an error, but to our code that just means that "no, that isn't a TTY". (And thus, your opened file will automatically end up buffered. The flow here is a good default, for each of the cases it is sussing out.)

Re: Not knowing the /proc file system

#66

Post by crabbone »

twic wrote: post #51
Galanwe wrote: post #8

Earlier quoted context omitted.

I generally agree with your comment, especially coming from OpenBSD where /proc does not exist, for reasons you mentioned, as well as a purely practical one: let's not pressure the VFS for no reason. I think the biggest issue with /proc at the moment is that it leaks in core Linux components, and getting rid of it is becoming impossible. Some years ago I worked on adding `$ORIGIN` rpath support in OpenBSD and looked…

> One thing though, you focus a lot on syscalls, but I think the challenge in recent years has been to find new ways to have kernel userspace interactions _outside_ of syscall, which are cumbersome to use, rigid in structure, and practically speaking, there are only so many entries you can store in an IDT. There's always ioctls. You could have something a lot like /proc and /sys, with a directory hierarchy modelling…

> There's always ioctls.

Not good. Not so long ago I ran into a problem with mdadm which instead of reading sysfs used ioctls which simply didn't carry enough information for it to function properly. So, it still has bugs because of that, but it being written in C, reading from a file and parsing stuff from a string seems to cause a lot of melancholy in its maintainers, so the bug has been on slow burner for years now.

Re: Not knowing the /proc file system

#67

Post by goodthenandnow »

Galanwe wrote: post #8
Animats wrote: post #6

The Linux /proc "file system" is kernel to user space communication hammered into the wrong form because "everything is a file". /proc is a system call with a fake file system API, and this matters. The sample code won't work reliably, because it assumes that the "files" won't change while being read. If you read /proc, you must "read" each file with one unbuffered kernel read to be free of race conditions. See [1].…

I generally agree with your comment, especially coming from OpenBSD where /proc does not exist, for reasons you mentioned, as well as a purely practical one: let's not pressure the VFS for no reason. I think the biggest issue with /proc at the moment is that it leaks in core Linux components, and getting rid of it is becoming impossible. Some years ago I worked on adding `$ORIGIN` rpath support in OpenBSD and looked…

> One thing though, you focus a lot on syscalls, but I think the challenge in recent years has been to find new ways to have kernel userspace interactions _outside_ of syscall, which are cumbersome to use, rigid in structure, and practically speaking, there are only so many entries you can store in an IDT.

Today there is another option for doing the kernel-userspace communication: using an io_uring like interface. Yes, it's a syscall, but it's one with enough extensibility and performance characteristics. It's extensible because it's close to an IPC with the kernel (and it could be changed to be a new IPC mechanism for process-to-process communication if it isn't already). And it's performant because it's based in shared memory and is assyncronous by default.

Re: Not knowing the /proc file system

#68

Post by Galanwe »

marcosdumay wrote: post #46
Animats wrote: post #6

The Linux /proc "file system" is kernel to user space communication hammered into the wrong form because "everything is a file". /proc is a system call with a fake file system API, and this matters. The sample code won't work reliably, because it assumes that the "files" won't change while being read. If you read /proc, you must "read" each file with one unbuffered kernel read to be free of race conditions. See [1].…

> The Linux kernel has no standard mechanism for delivering a variable-sized result from a system call. Maybe we can expand the userspace networking interfaces, and make a fully featured messaging system. Some day Linux may even reach feature parity with L4!

That is netlink.

https://man7.org/linux/man-pages/man7/rtnetlink.7.html

Re: Not knowing the /proc file system

#69

Post by crabbone »

9dev wrote: post #2

Tangent: Looking at those code samples, I wonder whether coming up with the shortest possible, most cryptic variable names is somewhat of a sport amongst C developers. Are you guys still coding in Notepad and need to conserve keystrokes, or where does that reluctance to use proper names come from?

You should try Prolog. That's where single-letter variables are really, really common.

C - not so much. In C you are more likely to see acronyms. It's also interesting that for some reason, macro names are spelled out in full, but function names are abbreviated. So, typical C looks like:

   LONG_SCREAMING_MACRO_NAME_(uh, oh);

Re: Not knowing the /proc file system

#70

Post by viraptor »

> lseek(3, 0, SEEK_CUR)

This part can determine if the descriptor is seekable or not. It's a noop if it is and returns an error if it isn't.

> It was quite clear from the strace output that the overhead of the interpreter costs practically the same as running the program itself.

This kind of idea keeps repeating and... it's misplaced. You can't use a high level API like glob and expect it will do the same minimum of work as your trivial implementation. This has nothing to do with the interpreter itself.

Post reply on HN