Pipe Viewer

1. things where I'd be paranoid about pv(1) itself becoming the bottleneck in the pipeline — e.g. dd(1) of large disks where I've explicitly set a large blocksize and set conv=idirect/odirect, to optimize throughput.

2. things where the program has some useful cleverness I rely on that requires being fed by a named file argument, but behaves a lot less intelligently when being fed from stdin — e.g. feeding SQL files into psql(1).

3. things where the program, even while writing to stdout, also produces useful "sampled progress" informational messages on stderr, which I'd like to see; where pv(1) and this output logging would fight each-other if both were running.

4. things where there's no clean place to insert pv(1) anyway — mostly, this comes up for any command that manages jobs itself in order to do things in parallel, e.g. any object-storage-client mass-copy, or any parallel-rsync script. (You'd think these programs would also report global progress, but they usually don't!)

I could see pv(1) being fixed to address case 3 (by e.g. drawing progress while streaming stderr-logged output below it, using a TUI); but the other cases seem to be fundamental limitations.

Personally, when I want to observe progress on some sort of operation that's creating files (rsync, tar/untar, etc), here's what I do instead: I run the command-line, and then, in a separate terminal connected to the machine the files are being written/unpacked onto, I run this:

    # for files
    watch -n 2 -- ls -lh $filepath

    # for directories
    watch -n 4 -- du -h -d 0 $dirpath

Doing things this way doesn't give you a percentage progress, but I find that with most operations I already know what the target's goal size is going to be, so all I really need to know is the size-so-far. (And pv(1) can't tell you the target size in many cases anyway.)

tar -cf - some/dir | ssh remote 'cd /place/to/go && tar -xvf -'