71–80 of 178 posts
Don't cat a file and pipe it into pv. Use "pv file" as a replacement for "cat file" and it will show you the progress as a percentage. When it's in the middle of a pipeline, it doesn't know the total size (unless you tell it with -s), so it can only show the throughput.
-> links to wikipedia page with direct discription of lineage back to 5th ed research unix
"That weird bs=4M argument in the dd version isn’t actually doing anything special—all it’s doing is instructing the dd command to use a 4 MB buffer size while copying. But who cares? Why not just let the command figure out the right buffer size automatically?"
Um -
a) it is 'doing the special thing' of changing the block size (not buffer size)
b) Because the command probably doesn't figure out the right size automatically, much like your 'cat' example above which also doesn't
c) And this can mean massive performance differences between invocations
> Another reason to prefer the cat variant is that it lets you actually string together a normal shell pipeline. For instance, if you want progress information with cat you can combine it with the pv command
Umm:
dd if=file bs=some-optimal-block-size | rest-of-pipeline
>If you want to create a file of a certain size, you can do so using other standard programs like head. For instance, here are two ways to create a 100 MB file containing all zeroes:
$ uname -sr
OpenBSD 6.0
$ head -c 10MB /dev/zero
head: unknown option -- c
usage: head [-count | -n count] [file ...]
$ man head |sed -ne 47,51p
HISTORY
The head utility first appeared in 1BSD.
AUTHORS
Bill Joy, August 24, 1977.
$ sed -ne 4p /usr/src/usr.bin/head/head.c
* Copyright (c) 1980, 1987 Regents of the University of California.
> So if you find yourself doing that a lot, I won’t blame you for reaching for dd. But otherwise, try to stick to more standard Unix tools.
Like 'pv'?
edit: added formatting, sector size note, head manpage/head.c stuffs.. apologies.
Dude's missing an important point: If you mess up the syntax on a dd invocation, a nice thing happens: nothing. Use a shell command and pipes, and your command better be perfect before you hit return.
Usually it's not past the easily-reproducable system partition yet or on a data disk that is backed up regularly so I can recover in an hour or so..
I think dd is primarily so popular because it is used in mostly dangerous operations. Sure, using cat makes logicial sense, but if we are talking about writing directly to disk devices here I'll trust the command I read from the manual and not explore commands I think would work. dd's "highly nonstandard syntax" comes from the JCL programming language, but it's really just another tool to read and write files. At the…
There's one good (?) reason to use dd with devices: it specifies target in the same command. For devices, writing to them usually requires root privileges, so it's easy to: sudo dd .... of=/dev/... But there's no trivial cat equivalent: sudo cat ... > target Will open target as your current user anyway. You can play around with tee and redirection of course. But that's getting more complicated than the original.
sudo (cat ... > target) //SYSPRINT DD SYSOUT=*
//SYSLIN DD DSN=&&OBJAPBND,
// DISP=(NEW,PASS),SPACE=(TRK,(3,3)),
// DCB=(RECFM=FB,LRECL=80,BLKSIZE=3200),
// UNIT=&SAMPUNIT
//SYSLIB DD DSN=SYS1.MACLIB,DISP=SHR
//SYSIN DD DSN=&SAMPLIB(IEWAPBND),DISP=SHRI think dd is primarily so popular because it is used in mostly dangerous operations. Sure, using cat makes logicial sense, but if we are talking about writing directly to disk devices here I'll trust the command I read from the manual and not explore commands I think would work. dd's "highly nonstandard syntax" comes from the JCL programming language, but it's really just another tool to read and write files. At the…
I always thought dd stood for disk destroyer, only ever used it for making low level copies of whole disks or shredding them with if=/dev/random. This thread has been informative and terrifying as I learn cat and cp are every bit as dangerous as dd! I never would expect something like cp xxx /dev/sda to actually work. Thinking about it, why should cp even support something like that? I'll copy files but I'll also DES…
It better :-)
But it all comes from the unix idea of everything is a file.
I think dd is primarily so popular because it is used in mostly dangerous operations. Sure, using cat makes logicial sense, but if we are talking about writing directly to disk devices here I'll trust the command I read from the manual and not explore commands I think would work. dd's "highly nonstandard syntax" comes from the JCL programming language, but it's really just another tool to read and write files. At the…
I always thought dd stood for disk destroyer, only ever used it for making low level copies of whole disks or shredding them with if=/dev/random. This thread has been informative and terrifying as I learn cat and cp are every bit as dangerous as dd! I never would expect something like cp xxx /dev/sda to actually work. Thinking about it, why should cp even support something like that? I'll copy files but I'll also DES…
That's the beauty of Unix.
Everything is a file. Thus every program that can work with files, can in fact work with everything.
It's actually very liberating.
Don't cat a file and pipe it into pv. Use "pv file" as a replacement for "cat file" and it will show you the progress as a percentage. When it's in the middle of a pipeline, it doesn't know the total size (unless you tell it with -s), so it can only show the throughput.
Actually, that's not completley true. pv will detect the file size if you use the shell to read a file into it, like pv < file.
pv file.bin | dd of=/dev/sdb