I don't always backup an entire system with 'dd', but when I do: dd if=/dev/da0 | ssh user@rsync.net "dd of=backup.dd" or maybe: pg_dump -U postgres db | ssh user@rsync.net "dd of=db_dump"
Unless you have a really big upload pipe, better use ssh -C.
A snapshot of your computer with dd, pv and gzip - Part 1
21–30 of 34 posts
Re: A snapshot of your computer with dd, pv and gzip - Part 1
#22This is an incredibly naive way of backing up your system. I use rsync to back up certain folders. I'm OK with re-installing Linux if my hard drive crashes.
It's not naive at all. There are a plethora of situations/reasons where one may want or need a block level image backup.
For example...
Re: A snapshot of your computer with dd, pv and gzip - Part 1
#23This is an incredibly naive way of backing up your system. I use rsync to back up certain folders. I'm OK with re-installing Linux if my hard drive crashes.
It's not naive at all. There are a plethora of situations/reasons where one may want or need a block level image backup.
Like when?
Scenario 1: I boot my PC up from a live CD do dd clone the main drive, since it's a bad idea to try to do it online. I create a giant, 500 gigabyte file containing every bit on my main hard drive. This takes 20 hours.
A month later my hard drive crashes and I want to restore it. I put in new hard drive, boot from live CD, and write my giant 500 gigabyte file onto the new drive and reboot. This takes 18 hours. Voila, I have my setup... from a month ago.
Scenario 2: I have a cron job to run rsync on a few folders that I care about. Rsync copies them over the network to my NAS. It runs every 12 hours and only copies the diffs. My main drive crashes. I put in a new one, re-install Linux (good opportunity to upgrade distributions), and sync my data back over. The whole process takes 2 hours.
So yeah. Incredibly naive. Sorry.
Re: A snapshot of your computer with dd, pv and gzip - Part 1
#24Pigz looks nice. First I've ever heard of it. I just use BackInTime though, like TimeMachine I guess. It's really, really useful. http://backintime.le-web.org/screenshots/
Looks like a graphical rsnapshot. Is this it?
Re: A snapshot of your computer with dd, pv and gzip - Part 1
#25Hi! Author of this article here,
I've done something completely silly. I originally wrote this site on github, but then moved it over to a new host on its own domain and totally forgot about the old one.
If you like this article, please check it out here at:
http://allgood38.io/a-snapshot-of-your-computer-with-dd-pv-a...
Re: A snapshot of your computer with dd, pv and gzip - Part 1
#26Earlier quoted context omitted.
It's not naive at all. There are a plethora of situations/reasons where one may want or need a block level image backup.
For example...
Well, it provides a perfect copy of the drive, not just a copy on another file system.
I've actually been able to restore to a drive, and when it boots its as though nothing ever happened. Plus its fast.
Its eerie to turn the computer on after knowing you formatted the drive for whatever reason, and your desktop pops up again.
Re: A snapshot of your computer with dd, pv and gzip - Part 1
#27Earlier quoted context omitted.
For example...
Well, it provides a perfect copy of the drive, not just a copy on another file system. I've actually been able to restore to a drive, and when it boots its as though nothing ever happened. Plus its fast. Its eerie to turn the computer on after knowing you formatted the drive for whatever reason, and your desktop pops up again.
It stops being eerie after a while. Several months ago, my brother was putting together a computer and wanted to test it (ie. the motherboard) before attaching any peripherals (other than the monitor). Obviously, installing the OS on the HD to do this was out of the question, so I just copied my computer onto a thumb-drive and used that. I didn't dd it, but just copied my filesystem over and did a grub-install.
Re: A snapshot of your computer with dd, pv and gzip - Part 1
#28Hmm, I'm a bit wary of this kind of backup, because it seems like it's nontrivial to recover part of your data if half of the backup goes bad. While the old fashioned rsync/copy all your files may be less efficient, you'll still have whatever it managed to copy if something goes wrong partway through. This reminds be of doing a tar archive of all your files--if the archive is corrupted during that process, what then?
I use clonezilla to image the whole hard drive including OS once I have a 'clean' setup with an empty home and all my software installed.
Then I use rsync to copy all my 'stuff' back off a large external hard drive to the 'clean' installation. I try not to change OS to often :-)
Subsequently I use rsync with --delete set to sync the home drive only to an external drive of similar capacity to the computer drive for daily changes. This drive is ex4 and preserves ownership and permissions, and includes dotfiles.
Once a week there is rsync without --delete to a large external hard drive as 2nd backup. This drive is NTFS and does not preserve permissions and excludes dotfiles.
I have a free dropbox account for really crucial 'working' files. That gets rsync'd to a FAT32 stick each day using the 'time window' command so it does not write everything each time.
Basically the clonzilla whole drive backup is just to save time/bandwidth with re-installing and setting up. The rsync backup to the large drive is file readable on work PCs that use Windows should disaster strike. The 'sync' to the small hard drive is readable at file level on anything I can boot from a live CD.
Now, a bootable whole disc image would be of interest now and again.
Re: A snapshot of your computer with dd, pv and gzip - Part 1
#29I don't always backup an entire system with 'dd', but when I do: dd if=/dev/da0 | ssh user@rsync.net "dd of=backup.dd" or maybe: pg_dump -U postgres db | ssh user@rsync.net "dd of=db_dump"
Unless you have a really big upload pipe, better use ssh -C.
I don't know if ssh -C is strong enough these days. I use pigz or maybe pxz -1.
Re: A snapshot of your computer with dd, pv and gzip - Part 1
#30Hmm, I'm a bit wary of this kind of backup, because it seems like it's nontrivial to recover part of your data if half of the backup goes bad. While the old fashioned rsync/copy all your files may be less efficient, you'll still have whatever it managed to copy if something goes wrong partway through. This reminds be of doing a tar archive of all your files--if the archive is corrupted during that process, what then?
Well actually, from a recovery point of view, you get some options. You can mount the image as a block device and perform operations on it. Say the image was corrupted, you could always try to mount the specific partition and run a fsck utility on it.