Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Progress – A Tool to Monitor Progress for Commands in Linux (2016)

tecmint.com

Post reply on HN
Print view View original

11–20 of 42 posts

Re: Progress – A Tool to Monitor Progress for Commands in Linux (2016)

#11

Post by anitil »

znpy wrote: post #6

I know what I am about to say is an absolutely ugly hack, but... If you cannot install non-standard software (example: customer owner, company operated RHEL boxes) you can get an idea of the current progress of cp/tar and similar programs by inspecting file-descriptor files in /proc. Assuming that you have a process (like cp) with pid 1234, then in /proc/1234/fd you will find files with a number (corresponding to the…

This is just fantastic. I had all the bits of information in my head to be able to do this, but never connected the dots.

Not ugly at all!

Re: Progress – A Tool to Monitor Progress for Commands in Linux (2016)

#12

Post by blibble »

paulddraper wrote: post #5

Usually I have used pv. https://linux.die.net/man/1/pv cp pv source > target tar tar czf images.tar.gz image1 image2 image3 | pv > target gzip pv source | gzip > target.gz # or gzip

you can give it a pid too

           -d PID[:FD], --watchfd PID[:FD]
              Instead of transferring data, watch file descriptor FD  of  process  PID,
              and  show  its progress.  The pv process will exit when FD either changes
              to a different file, changes read/write mode, or is  closed;  other  data
              transfer  modifiers  -  and  remote  control  - may not be used with this
              option.

              If only a PID is specified, then that process will be  watched,  and  all
              regular  files  and  block devices it opens will be shown with a progress
              bar.  The pv process will exit when process PID exits.

Re: Progress – A Tool to Monitor Progress for Commands in Linux (2016)

#13

Post by Waterluvian »

znpy wrote: post #6

I know what I am about to say is an absolutely ugly hack, but... If you cannot install non-standard software (example: customer owner, company operated RHEL boxes) you can get an idea of the current progress of cp/tar and similar programs by inspecting file-descriptor files in /proc. Assuming that you have a process (like cp) with pid 1234, then in /proc/1234/fd you will find files with a number (corresponding to the…

This is what I call a "22/7" solution. Close enough to be useful but you'd never use it formally. This is really great to have in my back pocket the next time a long running copy process feels hanged (often via. Ansible) and I just want a sense of, "okay are we minutes, hours, or days away from being done?"

Re: Progress – A Tool to Monitor Progress for Commands in Linux (2016)

#15

Post by michaelmior »

blibble wrote: post #12
paulddraper wrote: post #5

Usually I have used pv. https://linux.die.net/man/1/pv cp pv source > target tar tar czf images.tar.gz image1 image2 image3 | pv > target gzip pv source | gzip > target.gz # or gzip

you can give it a pid too -d PID[:FD], --watchfd PID[:FD] Instead of transferring data, watch file descriptor FD of process PID, and show its progress. The pv process will exit when FD either changes to a different file, changes read/write mode, or is closed; other data transfer modifiers - and remote control - may not be used with this option. If only a PID is specified, then that process will be watched, and all re…

Cool! I love pv and it's one of my favourite utilities but I never knew about this option. Thanks :)

Re: Progress – A Tool to Monitor Progress for Commands in Linux (2016)

#16

Post by newscracker »

paulddraper wrote: post #5

Usually I have used pv. https://linux.die.net/man/1/pv cp pv source > target tar tar czf images.tar.gz image1 image2 image3 | pv > target gzip pv source | gzip > target.gz # or gzip

I always use pv when I want to monitor a long running operation, mainly because it shows the rate of progress as well as the ETA for the operation (of course, this is an estimate, but it's still useful).

'man pv' is a short read and is something anyone who want to monitor such things should read.

Re: Progress – A Tool to Monitor Progress for Commands in Linux (2016)

#17

Post by progress-code »

znpy wrote: post #6

I know what I am about to say is an absolutely ugly hack, but... If you cannot install non-standard software (example: customer owner, company operated RHEL boxes) you can get an idea of the current progress of cp/tar and similar programs by inspecting file-descriptor files in /proc. Assuming that you have a process (like cp) with pid 1234, then in /proc/1234/fd you will find files with a number (corresponding to the…

  #!/bin/sh
  
  for pid in $@; do
  	cd /proc/"$pid"/fd || continue
  	
  	for fd in *; do
  		file=$(readlink "$fd");
  		if [ ! -z "$file" ]; then
  			size=$( (du -b "$file" 2>/dev/null || echo 0) | awk '{print $1}')
  			if [ "$size" = "0" ]; then
  				continue
  			fi
  			
  			pos=$(

Re: Progress – A Tool to Monitor Progress for Commands in Linux (2016)

#18

Post by awinter-py »

zooming out, progress in the cloud is something that should be easier

plenty of our headless jobs know exactly where they are and how big the job is. There should be a standard dash for getting this.

even things like hadoop & luigi that have dashboards built in aren't great at progress & ETA

we should surface & store stats from any long-running loop. Further benefit that you can use prev runtime to properly schedule the next run (or run more sophisticated feedback & alerting).

Re: Progress – A Tool to Monitor Progress for Commands in Linux (2016)

#19

Post by glandium »

znpy wrote: post #6

I know what I am about to say is an absolutely ugly hack, but... If you cannot install non-standard software (example: customer owner, company operated RHEL boxes) you can get an idea of the current progress of cp/tar and similar programs by inspecting file-descriptor files in /proc. Assuming that you have a process (like cp) with pid 1234, then in /proc/1234/fd you will find files with a number (corresponding to the…

That's exactly what the tool in the link does.

Re: Progress – A Tool to Monitor Progress for Commands in Linux (2016)

#20

Post by kjeetgill »

znpy wrote: post #6

I know what I am about to say is an absolutely ugly hack, but... If you cannot install non-standard software (example: customer owner, company operated RHEL boxes) you can get an idea of the current progress of cp/tar and similar programs by inspecting file-descriptor files in /proc. Assuming that you have a process (like cp) with pid 1234, then in /proc/1234/fd you will find files with a number (corresponding to the…

I often use the basic: $ watch -n.2 "du -sb src dest" I suppose you could add awk to get a percentage.
Post reply on HN