Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Not knowing the /proc file system

admccartney.mur.at

Post reply on HN
Print view View original

81–90 of 111 posts

Re: Not knowing the /proc file system

#81

Post by chaps »

/proc is amazing once you get the hang of it and get a good understanding of what's all in there. Especially if you're doing low level performance tuning.

It's particularly helpful in larger infrastructures where tool variability means differences in available commands, their output, and cli options. I'm sure /proc iteration has its own issues of variability across large infrastructres, but I haven't seen it. It's a fairly consistent API. Or at least it was, since I haven't touched a large infrastructure in some time.

When I got tired of `lsof` not being installed on hosts (or when its `-i` param isn't available) I ended up writing a script [1] that just iterates through /proc over ssh and grabs all inet sockets, environment variables, command line, etc from a set of hosts. Results in a null-delimited output that can then be fed into something like grafana to create network maps. Biggest problem with it is the use of pipes means all cores go to 100% for the few seconds it takes to run.

[1] https://github.com/red-bin/lsofer

Re: Not knowing the /proc file system

#82

Post by chasil »

tczMUFlmoNk wrote: post #61
chasil wrote: post #54

Here is a little shell script to print out what is easily seen in /proc/*/cmdline. This requires a GNU xargs that supports NULL termination (or something compatible); as I understand it, this cannot be done with POSIX tools. $ cat shps #!/bin/dash for path in /proc/*/cmdline do p=${path#*/} p=${p#*/} p=${p%/*} case "$p" in *[!0-9]*) continue;; esac c="$(xargs -0 echo

> This requires a GNU xargs that supports NULL termination (or something compatible); as I understand it, this cannot be done with POSIX tools. You can do this with `tr`, right? #!/bin/dash for path in /proc/*/cmdline do p=${path#*/} p=${p#*/} p=${p%/*} case "$p" in *[!0-9]*) continue;; esac c="$(tr '\0' ' ' Those escape sequences are covered in the standard: https://pubs.opengroup.org/onlinepubs/9699919799/utilities…

I had read elsewhere that POSIX tr was not capable of this, but it appears that I am wrong.

Re: Not knowing the /proc file system

#83

Post by 9dev »

enriquto wrote: post #76
Terr_ wrote: post #71

Earlier quoted context omitted.

That one lands near the margin-of-error for my sarcasmometer, but either way I'd like to emphasize (non-ironically) that the terse form is indeed very efficient... for someone repeatedly writing/copying it down by hand using a quill and ink. However that particular use-case has become dramatically less significant.

Heh. No sarcasm at all in my comment! (but I like to write in an over the top way...) Terse notation has nothing to do with manual handwriting. Modern math books and articles are still written by computer using a very terse symbolic notation, which has been developed during the last six centuries. Originally, the symbols +, = were shorthand abbreviations of Latin words. I guess computer scientists want to re-invent e…

Spoken like someone who doesn’t need or care to cooperate with others on the same piece of code, doesn’t work with junior developers, and/or works on small code bases or scientific code exclusively.

Mathematicians seem to miss the difference between math and code quite often. The former provides the solution to a well-understood problem in a straightforward manner. The latter transports an abstract concept, a plan, a state of thought to the reader. A neat side effect is making computers go beep. In that context, being as clear as possible is really important.

Re: Not knowing the /proc file system

#84

Post by mzs »

williamcotton wrote: post #32

This is running in a barebones Ubuntu container on my MacBook as BSDs don't use /proc: root@74c03a282fbe:/# ed a ls /proc/[0-9]/status | xargs -n 1 cat | awk '/^Name:/ { name = $2 } /^Pid:/ { pid = $2 } END { print "cmd: " name ", pid: " pid }' . w prc.sh 132 q root@74c03a282fbe:/# chmod +x ./prc.sh root@74c03a282fbe:/# hyperfine --warmup=100 "./prc.sh" Benchmark 1: ./prc.sh Time (mean ± σ): 2.2 ms ± 0.3 ms [User: 1.…

I like your style, but it's not quite right:

  $ echo /proc/[0-9]/status | wc
        1       7     105
  $ echo /proc/[1-9]*/status | wc
        1     483    8778
It also will sporadically print error messages due to all the race conditions. Here's my stab at it:

  $ cat dumbps 
  #!/bin/sh
  
  case $# in
    1)
   ;;
    *)
   echo "Usage: dumbps user" >&2
   exit 2
   ;;
  esac
  
  2>&- find /proc -mindepth 2 -maxdepth 2 -type f -name status -user "$1" | awk '{
   while ((getline li  0) {
    if (li ~ /^Name:/) {
     split($0, fn, "/")
     print fn[3], substr(li, 6)
     break
    }
   }
  
   close($0)
  }'
  $ ./dumbps "$USER" | grep -w 'bash$'
  1616678  bash
  $ ./dumbps 0 | grep -w 'systemd$'
  1  systemd
  $
It's pretty fast too:

  $ ls -ld /proc/[1-9]*/status | awk '$3 == "root"' | wc
      445    4005   29367
  $ time ./dumbps >/dev/null            
  Usage: dumdps user
  
  real 0m0.001s
  user 0m0.000s
  sys 0m0.001s
  $

Re: Not knowing the /proc file system

#85

Post by [deleted] »

spacechild1 wrote: post #75
mananaysiempre wrote: post #40

> I’m using a custom function for reading lines from a file. [Listing: fgetLine()] It’s probably easier to just use getline(), it does basically the same thing and is in POSIX.1-2008[1]. [1] https://pubs.opengroup.org/onlinepubs/9699919799.2018edition...

AFAICT, getline() is not available on Windows, neither with MSVC nor MinGW. The actual solution is to use C++ instead of C :)

[deleted]

Re: Not knowing the /proc file system

#86

Post by fsckboy »

Animats wrote: post #6

The Linux /proc "file system" is kernel to user space communication hammered into the wrong form because "everything is a file". /proc is a system call with a fake file system API, and this matters. The sample code won't work reliably, because it assumes that the "files" won't change while being read. If you read /proc, you must "read" each file with one unbuffered kernel read to be free of race conditions. See [1].…

you're being too negative, it's as if you don't understand "worse is better".

>If you read /proc, you must "read" each file with one unbuffered kernel read to be free of race conditions.

you make this sound like it's some arcane and tricky incantation; it's totally normal, like saying "to walk, put one foot in front of the other". It's unix file I/O 101, bufferless file operations are always atomic, it's a feature. There is absolutely no problem making atomic reads from files, it's a massive fucking feature.

The point is not that this is the best conceivable way to get this information, but that by making this information available this way by default, you can write and leverage all sorts of shell scripts while you are noodling at your keyboard. You don't very often need to turn your keyboard noodling into a 6-sigma 5-nines server, but if you know how to rip through this type of keyboard noodling, I'd trust you a lot more when it comes to uptime than all the people who show up at meetings ranting about 6-sigma and 5-nines while they cite Dave Cutler.

btw linux the /proc text file tree has already been supplanted by a binary "restatement" of the same and more data. I think this is more in keeping with the standard unix/solaris /proc approach, but it's not something I've had a need to look into much.

Re: Not knowing the /proc file system

#87

Post by a1369209993 »

xyzelement wrote: post #72
School-Cotton wrote: post #62

Earlier quoted context omitted.

CNLabelContactRelationBiaoMei would have been a lot better. People can look up what that means when they need to.

I haven't been a dev in a bit, but I'd say between having a longer variable name, and having to crack open the fucking dictionary.... I have a clear preference.

Exactly: one option is a single-time, fixed cost (single-time-ever if you write it as:

  CN_LCR_BiaoMei = 77,
  // either mother's sibling's daughter or father's sister's daughter
  // aka female cousin involving at least one female parent
), while the other adds a constant tax on all uses of the variable (well, constant in this case) in perpetuity.

Re: Not knowing the /proc file system

#88

Post by kjs3 »

9dev wrote: post #2

Tangent: Looking at those code samples, I wonder whether coming up with the shortest possible, most cryptic variable names is somewhat of a sport amongst C developers. Are you guys still coding in Notepad and need to conserve keystrokes, or where does that reluctance to use proper names come from?

Sure...all us oldsters should be using things like ThisIsAVariable_i_like_it_very_much and spend a few hours making sure our bloated IDE highlights the variables we really like in mauve. Bonus points for using ChatGPT to generate the variable names.

Re: Not knowing the /proc file system

#89

Post by zokier »

Galanwe wrote: post #56
twic wrote: post #51

Earlier quoted context omitted.

> One thing though, you focus a lot on syscalls, but I think the challenge in recent years has been to find new ways to have kernel userspace interactions _outside_ of syscall, which are cumbersome to use, rigid in structure, and practically speaking, there are only so many entries you can store in an IDT. There's always ioctls. You could have something a lot like /proc and /sys, with a directory hierarchy modelling…

Agree, ioctls are a good replacement of /proc for exposing simple, structured information to/from the kernel. It is still not perfect for all use cases though, especially because it forces you to operate in a _polling_ fashion. Programs like top/iotop/htop are a good example (also, who is not guilty of abusing one liners a-la `watch -n1 cat /proc/meminfo`?). Instead of polling /proc or ioctls, you would really like t…

Netlink is pretty much as good as it gets here. It is explicitly designed as improvement over ioctl mess

> Netlink is often described as an ioctl() replacement. It aims to replace fixed-format C structures as supplied to ioctl() with a format which allows an easy way to add or extended the arguments.

Post reply on HN