Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Cult of DD

eklitzke.org

Post reply on HN
Print view View original

81–90 of 178 posts

Re: The Cult of DD

#81

Post by angry_octet »

apostacy wrote: post #71
electrum wrote: post #59

Don't cat a file and pipe it into pv. Use "pv file" as a replacement for "cat file" and it will show you the progress as a percentage. When it's in the middle of a pipeline, it doesn't know the total size (unless you tell it with -s), so it can only show the throughput.

Actually, that's not completley true. pv will detect the file size if you use the shell to read a file into it, like pv < file.

At first I thought, no, that's not possible. Then I thought, no, they wouldn't do THAT would they?

But I guess they do...

http://stackoverflow.com/questions/1734243/in-c-how-do-i-pri...

I've seen that kind of brokenness from programs trying to find their binary image on disk. Don't do it, it's bad.

Re: The Cult of DD

#82

Post by angry_octet »

AdamJacobMuller wrote: post #13
knz42 wrote: post #7

What about the `seek` argument which skips over some blocks at the beginning but still allocates them (unix "holes")? Also note that there are still unix systems out there which do not support byte-level granularity of access to block devices. On those devices you must actually use a buffer of exactly the size of the blocks on the device. Heck, linux was like this until at least v2.

Also keep in mind that specifying the block size can be important, especially for efficiently reading data. standard shell tools don't just "figure it out" automatically. They guess, and sometimes those assumptions can be incorrect resulting in lower (orders of magnitude) performance.

Very useful when dealing with raid (make the blocks stripe sized) or tape (512 byte) or esoteric devices.

An essential tool for low level repair, like when you can guess the partition table values but there is no partition table anymore.

Re: The Cult of DD

#83

Post by kam »

angry_octet wrote: post #81
apostacy wrote: post #71

Earlier quoted context omitted.

Actually, that's not completley true. pv will detect the file size if you use the shell to read a file into it, like pv < file.

At first I thought, no, that's not possible. Then I thought, no, they wouldn't do THAT would they? But I guess they do... http://stackoverflow.com/questions/1734243/in-c-how-do-i-pri... I've seen that kind of brokenness from programs trying to find their binary image on disk. Don't do it, it's bad.

It doesn't need to hunt for the directory entry, just needs to call `fstat()` on the stdin file descriptor.

Re: The Cult of DD

#85

Post by kev009 »

Ignorance on the blocksize arg.

Also, I only need to remember one progress command for my entire operating system: control+t. I also get a kernel wait channel from that which is phenomenally pertinent to rapidly understanding and diagnosing what the heck a command is doing or why it is stuck.

I hate what Linux has done to systems software culture.

Re: The Cult of DD

#86

Post by angry_octet »

gfody wrote: post #74
chrisfosterelli wrote: post #20

I think dd is primarily so popular because it is used in mostly dangerous operations. Sure, using cat makes logicial sense, but if we are talking about writing directly to disk devices here I'll trust the command I read from the manual and not explore commands I think would work. dd's "highly nonstandard syntax" comes from the JCL programming language, but it's really just another tool to read and write files. At the…

I always thought dd stood for disk destroyer, only ever used it for making low level copies of whole disks or shredding them with if=/dev/random. This thread has been informative and terrifying as I learn cat and cp are every bit as dangerous as dd! I never would expect something like cp xxx /dev/sda to actually work. Thinking about it, why should cp even support something like that? I'll copy files but I'll also DES…

It isn't the same though, because not all files are non-seekable streams of bytes. In fact, most are seekable and possibly sparse.

What happens when you did a sparse file? And cp?

https://wiki.archlinux.org/index.php/sparse_file

C.f. fallocate(1,2)

Re: The Cult of DD

#87

Post by angry_octet »

kam wrote: post #83
angry_octet wrote: post #81

Earlier quoted context omitted.

At first I thought, no, that's not possible. Then I thought, no, they wouldn't do THAT would they? But I guess they do... http://stackoverflow.com/questions/1734243/in-c-how-do-i-pri... I've seen that kind of brokenness from programs trying to find their binary image on disk. Don't do it, it's bad.

It doesn't need to hunt for the directory entry, just needs to call `fstat()` on the stdin file descriptor.

Oops you're right, has size in the inode.

Re: The Cult of DD

#88

Post by [deleted] »

angry_octet wrote: post #81
apostacy wrote: post #71

Earlier quoted context omitted.

Actually, that's not completley true. pv will detect the file size if you use the shell to read a file into it, like pv < file.

At first I thought, no, that's not possible. Then I thought, no, they wouldn't do THAT would they? But I guess they do... http://stackoverflow.com/questions/1734243/in-c-how-do-i-pri... I've seen that kind of brokenness from programs trying to find their binary image on disk. Don't do it, it's bad.

[deleted]

Re: The Cult of DD

#89

Post by adrianratnapala »

angry_octet wrote: post #81
apostacy wrote: post #71

Earlier quoted context omitted.

Actually, that's not completley true. pv will detect the file size if you use the shell to read a file into it, like pv < file.

At first I thought, no, that's not possible. Then I thought, no, they wouldn't do THAT would they? But I guess they do... http://stackoverflow.com/questions/1734243/in-c-how-do-i-pri... I've seen that kind of brokenness from programs trying to find their binary image on disk. Don't do it, it's bad.

If all they want is the size of the file, then what is wrong with using fstat?

It's not the same thing as trying to walk the FS to look for the filename is silly.

Re: The Cult of DD

#90

Post by rwmj »

kam wrote: post #83
angry_octet wrote: post #81

Earlier quoted context omitted.

At first I thought, no, that's not possible. Then I thought, no, they wouldn't do THAT would they? But I guess they do... http://stackoverflow.com/questions/1734243/in-c-how-do-i-pri... I've seen that kind of brokenness from programs trying to find their binary image on disk. Don't do it, it's bad.

It doesn't need to hunt for the directory entry, just needs to call `fstat()` on the stdin file descriptor.

Unless the input file is a device, and then you need to call:

    ioctl(STDIN_FILENO, BLKGETSIZE64, &size)
Post reply on HN