Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I lost my data trying to back it up

hamy.io

Post reply on HN
Print view View original

1–10 of 126 posts

Re: I lost my data trying to back it up

#3

Post by kkarakk »

why would you try this out with "live" data? why not incrementally connect drives and do it that way?

i'm basically asking what the impetus was for "just going for it" which is probably going to be a reason that other people share...

Re: I lost my data trying to back it up

#5

Post by cm2187 »

That sounds like a very complicated way to do a full OS backup.

Dead easy, 10 min, can't fail alternative for home projects: use a VM instead, switch off the VM, copy the disk file, switch the VM back on. Use NVMe SSDs and depending on the size, you can probably do an automated daily backup with minimal downtime.

Re: I lost my data trying to back it up

#8

Post by arendtio »

Well, it is not like I didn't do any stupid things in the past, but booting into to a different OS (live system) with such a setup is obviously risky.

I am a bit conservative when it comes to backups and I do not trust myself very much. So I am doing backups on the block device level to avoid not copying certain file types, file attributes or whatever feature the filesystem in question posses.

In order to avoid, risky OS reboots, I am using lvm snapshots like this:

  # sync the filesystem to disk
  sync
  # create a snapshot with a 10GB buffer
  lvcreate -L10G -s -n "$snapshot" "/dev/vg/$lv"
  # write the backup to a backup disk
  dd if="/dev/vg/$snapshot" of="/mnt/backups/lvm-vg-${lv}_backup-$(date +%Y%m%d).dd" bs=64k
  # remove the snapshot
  lvremove -f "/dev/vg/$snapshot"
That method is certainly not perfect, as you are still taking the backup with a mounted FS in place and the snapshot size must be adapted to the use-case, but in the end, this is a method to create backups which are complete (independent of FS), easily accessible (e.g. mount via loop device) and do not interrupt the service.

Re: I lost my data trying to back it up

#9

Post by simonh »

There's now way he could have possible known simply booting the Ubuntu live disk would do this. He could have done that without any consideration to doing a backup. However regular tried and true backups are the way to go, preferably automated.

My worst horror story wasn't really mine. I did desktop support in my first job. The library had their own system and their own support arrangements but I had a good relationship with them and would advise from time to time. One day they said the PC running their archive management system wouldn't boot. Could I take a quick look before they called the company on their support contract? Sure enough the PC wouldn't boot, the hard drive had died. I asked if they had recent backups? Oh yes the backups were fine, in fact it had got much better recently. Before the backups used to take an hour, but for for the last few weeks it only took 2 minutes. They only had 2 weeks of tapes on rotation.

Needless to say, the backups hadn't been running at all and failed silently, probably due to early symptoms of the hard drive failure. They lost everything.

Re: I lost my data trying to back it up

#10

Post by olavgg »

Getting backup right is extremely important, that is why you need proper tools. For the last 10 years I have been using ZFS snapshots and replication. I keep 180 days with snapshots on the remote backup and 30 days for the local. I use it to take backup of virtual machines, different databases including "running", sql servers. And of course important files, documents and photos.

I've also configured a daily mail with a log about how the backup/snapshot/replication went. Even if it went fine. So if I don't get mail, I know something is up and can fix it asap.

I have never slept so well after doing this with ZFS, even backup for Windows SQL Server with iSCSI volumes works great!

More people should do backup with ZFS!

Post reply on HN