I lost my data trying to back it up • Hacker News Board

Re: I lost my data trying to back it up

#71

Post by sn » Thu, Jan 31, 2019, 5:59 PM UTC

The recommendation to only wipe the first and last part of the drive before reusing them is inadequate. You're best off wiping the entire drive with either zeros or hdparm security-erase-enhanced. The reason is that not all metadata is kept just at the beginning or end of the drive, especially if you have multiple partitions in the drive.

If that is untenable, mdadm has an option --zero-superblock, lvm has pvremove, and wipefs will remove file system signatures, which you can run before changing the storage layout.

Re: I lost my data trying to back it up

#72

Post by sn » Thu, Jan 31, 2019, 6:03 PM UTC

sokoloff wrote: post #36

cm2187 wrote: post #5

That sounds like a very complicated way to do a full OS backup. Dead easy, 10 min, can't fail alternative for home projects: use a VM instead, switch off the VM, copy the disk file, switch the VM back on. Use NVMe SSDs and depending on the size, you can probably do an automated daily backup with minimal downtime.

Anyone have a solution that can snapshot a running VM that has a PCI or PCIe card passed through to the guest OS? I use snapshots on ESXi, but they don't work for guests that have a card passed-through.

Are you asking for a recommendation on how to do this with ESXi or a recommendation for an alternative to ESXi?

Re: I lost my data trying to back it up

#73

Post by h4myio » Thu, Jan 31, 2019, 6:05 PM UTC

sn wrote: post #71

The recommendation to only wipe the first and last part of the drive before reusing them is inadequate. You're best off wiping the entire drive with either zeros or hdparm security-erase-enhanced. The reason is that not all metadata is kept just at the beginning or end of the drive, especially if you have multiple partitions in the drive. If that is untenable, mdadm has an option --zero-superblock, lvm has pvremove,…

Thanks for pointing this out.

Do you have an example (or rather source) of a RAID metadata outside of the first/last 1MiB of the drive?

Re: I lost my data trying to back it up

#74

Post by tbyehl » Thu, Jan 31, 2019, 6:05 PM UTC

If you have the resources to run your own backup server, I heartily recommend UrBackup[1]. It does Windows right by default. It can handle Linux ARM devices. Macs, too. Setup of the server and clients is super simple.

10/10, would install again.

[1] https://www.urbackup.org

Re: I lost my data trying to back it up

#75

Post by dang » Thu, Jan 31, 2019, 6:49 PM UTC

21 wrote: post #47

No mention of offsite backup. Article could have stopped there. More subtly, he is the epitome of what I call "cowboy programmer/sysadmin", who doesn't really know what he's doing, just always improvising the next step based on minimal research. Which is fine and productive, until you are dealing with critical data/systems. I've witnessed so many times during my career this kind of people royally fuck things up with…

Personal attacks are not ok here, regardless of how bad you feel someone is at backups.

Please review https://news.ycombinator.com/newsguidelines.html and follow the rules when posting here. We've already had to ask you many times. When accounts continue to violate the guidelines, we end up banning them, and I don't want to do that.

Re: I lost my data trying to back it up

#76

Post by sokoloff » Thu, Jan 31, 2019, 6:58 PM UTC

sn wrote: post #72

sokoloff wrote: post #36

Earlier quoted context omitted.

Anyone have a solution that can snapshot a running VM that has a PCI or PCIe card passed through to the guest OS? I use snapshots on ESXi, but they don't work for guests that have a card passed-through.

Are you asking for a recommendation on how to do this with ESXi or a recommendation for an alternative to ESXi?

Either is fine. If it's within-ESXi, that's a little easier, but I'm not tightly/inextricably wed to that hypervisor.

Re: I lost my data trying to back it up

#77

Post by mgsouth » Thu, Jan 31, 2019, 6:58 PM UTC

guitarbill wrote: post #67

NKosmatos wrote: post #27

Sorry for your loss. Here are my two cents... For complete system backup on Windows use Acronis [0], on MacOS use Carbon Copy Cloner [1] and on Linux Clonezilla [2]. Also never forget the 3-2-1 rule of backups [3], 3 copies, 2 local copies on different mediums and 1 copy offsite (cloud, remote) :-) [0] https://www.acronis.com/en-us/personal/computer-backup/ [1] https://bombich.com/ [2] https://clonezilla.org/ [3] htt…

For macOS, Time Machine can do automatic backups to a mounted NAS volume over WiFi. So my Macbook backs up to a ZFS 3-way mirror. Important files are also stored in DropBox. That's how I get relatively hassle free 3-2-1 backups. For Windows, a similar strategy is possible, but I consider my Windows box disposable (only gaming), so don't have much recent experience.

I stopped using TM. It had (maybe still does) the really nasty habit of corrupting the virtual backup volume when my laptop's connection to the server was interrupted. You know, like closing the lid and taking it in to work. The TM volumes were on a Netatalk server; _it_ had no problem maintaining a consistent wrapper volume (the files inside it, including the TM virtual containers, were always perfect), but TM apparently made/makes non-journaled, non-atomic internal writes.

Re: I lost my data trying to back it up

#78

Post by sbov » Thu, Jan 31, 2019, 7:01 PM UTC

I'm always paranoid about this stuff.

There's lots of fancy tech words in that article. Before you use ANY piece of backup tech you do not 100.0000000000% understand, back up your data with tech you COMPLETELY and UTTERLY understand how to use.

This likely means a simple copy to an external drive.

After that, set up your first automated backup solution. Test that it works.

After that, set up your second automated backup solution. Test that it works.

For the especially paranoid (me), you can still do manual backups using tech you completely and utterly understand.

Re: I lost my data trying to back it up

#79

Post by asaph » Thu, Jan 31, 2019, 7:12 PM UTC

And... I'm backing up my data right now...

Re: I lost my data trying to back it up

#80

Post by herpderperator » Thu, Jan 31, 2019, 7:35 PM UTC

You should always favor mdadm (software raid) over fake RAID solutions. Sometimes it's better to use this even over real RAID cards (which are proprietary) because you don't have to find an exact clone of the card if it fails; mdadm is all software. The only time it would be appropriate to use a real RAID card is when the performance gain is an absolutely necessary requirement, or if you need to expose a logical device to the BIOS or OS that maps to the RAID array without additional setup.