Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Cult of DD

eklitzke.org

Post reply on HN
Print view View original

111–120 of 178 posts

Re: The Cult of DD

#111

Post by arjie »

gkya wrote: post #106
cat199 wrote: post #72

"This is a strange program of obscure provenance that somehow, still manages to survive in the 21st century." -> links to wikipedia page with direct discription of lineage back to 5th ed research unix "That weird bs=4M argument in the dd version isn’t actually doing anything special—all it’s doing is instructing the dd command to use a 4 MB buffer size while copying. But who cares? Why not just let the command figure…

> pv (You probably know this as you use OpenBSD, but) something I really like about BSDs is that nost of the core commands respond to ^T with progress info ofsome kin, dd included.

Woah. That's neat. Thank you!

Re: The Cult of DD

#112

Post by lokedhs »

betaby wrote: post #29

Author is wrong bs IS useful, try to dd one hard drive to another without reasonable bs (1-8M) with and without and you will see a difference.

Try it on a tape device with the wrong value for bs and you might corrupt the output as a result.

Re: The Cult of DD

#114

Post by keithpeter »

CPLX wrote: post #100
ue_ wrote: post #23

Earlier quoted context omitted.

>One thing I'll often use dd for is recovering data from a failing drive. Funnily enough, I ended up using it to accidentally name the wrong drive in the argument, and lost years of photos, music, video etc. though I suppose I can't blame dd for that :)

I think there's a rule that you're not really qualified to discuss command line tools in public until you've used dd to inadvertently eradicate an entire partition.

Personally, I'd add the use of a trailing full-stop in an rsync command in the wrong directory (e.g. in /home/user) as an alternative qualification to your rule.

I now use full paths for destination as well as source.

Re: The Cult of DD

#115

Post by gaius »

rfw wrote: post #110
gaius wrote: post #105

Earlier quoted context omitted.

It makes a decent interview question tho', "explain the difference between cat file|./prog and ./prog <file". It doesn't even matter if they get it wrong, that they even know there is a difference is a very good sign.

I think most people wouldn't know the difference (I had no idea!) and the knowledge might fall into the realm of obscure trivia.

People who have come into SA work via being C programmers usually figure it out, they make the best SAs because they are mentally equipped to reason about a system from first principles.

Re: The Cult of DD

#116

Post by derefr »

miles wrote: post #99
cat199 wrote: post #72

"This is a strange program of obscure provenance that somehow, still manages to survive in the 21st century." -> links to wikipedia page with direct discription of lineage back to 5th ed research unix "That weird bs=4M argument in the dd version isn’t actually doing anything special—all it’s doing is instructing the dd command to use a 4 MB buffer size while copying. But who cares? Why not just let the command figure…

>> if you want progress information with cat you can combine it with the pv command Since coreutils-8.24[1], dd "accepts a new status=progress level to print data transfer statistics on stderr approximately every second." > Because the command probably doesn't figure out the right size automatically . . . this can mean massive performance differences between invocations For anyone who's wondering, here are two good t…

Or, before that, just send the dd process SIGINFO to get the same output printed once. `while :; do kill -INFO %1; sleep 1; done` if you want a "progress bar" of sorts.

As an aside: On BSDs (incl. macOS), SIGINFO is also able to be sent interactively by the line driver when you type ^T (like ^C sends SIGINT.) Kind of lame that Linux doesn't follow suit [or even have SIGINFO], or we'd see a lot more programs that build in useful "prod me for an update" hooks, the way they already have "prod me to reload my config" SIGHUP hooks.

Re: The Cult of DD

#117

Post by tankenmate »

cat199 wrote: post #72

"This is a strange program of obscure provenance that somehow, still manages to survive in the 21st century." -> links to wikipedia page with direct discription of lineage back to 5th ed research unix "That weird bs=4M argument in the dd version isn’t actually doing anything special—all it’s doing is instructing the dd command to use a 4 MB buffer size while copying. But who cares? Why not just let the command figure…

The other reason for dd existence is that back in the day if you wrote to a block device with a write(2) size of anything other than a multiple of the devices actual block size you'd get a EIO or a EINVAL. So the program you used to make sure the write(2) size was always correct was dd. In the old days some truly weird block devices would accept a write(2) that used only the exact block size, i.e. you could only read and/or write one block at once (if you tried to read more than one block of data you'd get back exactly one block regardless of how big your read buffer was). Old raw CD and WORM drives come to mind. Audio CDs for example had a 2,352 byte block size (after removing the CRC).

EDIT: fixed man section references

Re: The Cult of DD

#118

Post by floatboth »

GalacticDomin8r wrote: post #55
merlincorey wrote: post #30

Earlier quoted context omitted.

This is a good point as well. On BSD, we don't have `pv`, but we do have ^T. This will print some sort of status for just about any long running process. It prints very specialized status for certain programs aware of it.

pv is available on at least FreeBSD as a package like it is in Linux. I'd be quite shocked if the other BSD's didn't have a port of it as well. There is also mbuffer. SIGINFO works on gnu dd last I tried it.

Yeah, it's just that Linux doesn't even define SIGINFO

Re: The Cult of DD

#119

Post by floatboth »

schoen wrote: post #53
ocschwar wrote: post #48

Dude's missing an important point: If you mess up the syntax on a dd invocation, a nice thing happens: nothing. Use a shell command and pipes, and your command better be perfect before you hit return.

The biggest counterexample to this that some people have experienced is accidentally swapping if= and of=, thus backing up their target onto their source rather than vice versa.

Well that's not syntax…

Re: The Cult of DD

#120

Post by krylon »

gfody wrote: post #74
chrisfosterelli wrote: post #20

I think dd is primarily so popular because it is used in mostly dangerous operations. Sure, using cat makes logicial sense, but if we are talking about writing directly to disk devices here I'll trust the command I read from the manual and not explore commands I think would work. dd's "highly nonstandard syntax" comes from the JCL programming language, but it's really just another tool to read and write files. At the…

I always thought dd stood for disk destroyer, only ever used it for making low level copies of whole disks or shredding them with if=/dev/random. This thread has been informative and terrifying as I learn cat and cp are every bit as dangerous as dd! I never would expect something like cp xxx /dev/sda to actually work. Thinking about it, why should cp even support something like that? I'll copy files but I'll also DES…

> I'll also DESTROY YOUR SHIT if you say so

That's the Unix way: The customer, eh, user is always right.

I once wanted to clean up backup files created by emacs (they end in the tilde character) by typing "rm ~" - except what I did type was "rm ~".

(On the upside, I learned a valuable lesson that day.)

Post reply on HN