I Nearly Lost All of My Data • Hacker News Board

Re: I Nearly Lost All of My Data

#21

Post by jordache » Sat, Jan 19, 2019, 10:53 PM UTC

derekp7 wrote: post #13

The best solution I found as an alternative to an off-site copy is to set up a USB drive and something like a Raspberry Pi in my car. When I pull into the garage, the house server senses it and syncs new data to it. This can be supplemented further by having it auto sync to a computer in your office whenever you pull into work. Add in some monitoring so you can get a phone alert if any of the synced copies are more t…

how do you power the usb drive whilst it's syncing in your car, while parked in the garage?

What if the drive gets stolen?

Re: I Nearly Lost All of My Data

#22

Post by ocdtrekkie » Sat, Jan 19, 2019, 10:53 PM UTC

RcouF1uZ4gsC wrote: post #18

Couple of observations. First of all, given the price of storage, for backups, I don’t think anything other than mirroring makes sense. Just get 2 big hard drives for your NAS and set them up for mirroring. In the event of a filmier, you can read directly. Next, you don’t have a full backup without offsite storage. Even if it wasn’t a power surge, there could be flooding or fire at a single location. Always remember…

Indeed. I'd rather have a bunch of RAID 1s around than a RAID 5 I have to worry about rebuilding. I sync my files to two different RAID 1 setups in two different cities. No hardware failure has ever been worrying.

Re: I Nearly Lost All of My Data

#23

Post by ce4 » Sat, Jan 19, 2019, 10:54 PM UTC

zbuf wrote: post #16

I had a similar experience with a relative's Seagate NAS. Except the ext filesystem was unreadable because it used a different page size. Required some shenanigans in userland by thankfully I was able to recover the data. Seemed like a software fault on the box. The chassis had to be destroyed to remove the drive and it was interesting to see the warranty explicitly mention the customer was allowed to do this to reco…

Have you written up the solution or do you have some link? I have that same problem currently but havent had time to look at it yet. Thanks!

Re: I Nearly Lost All of My Data

#24

Post by gtirloni » Sat, Jan 19, 2019, 11:00 PM UTC

derekp7 wrote: post #13

The best solution I found as an alternative to an off-site copy is to set up a USB drive and something like a Raspberry Pi in my car. When I pull into the garage, the house server senses it and syncs new data to it. This can be supplemented further by having it auto sync to a computer in your office whenever you pull into work. Add in some monitoring so you can get a phone alert if any of the synced copies are more t…

Does the RaspberryPi have enough CPU for disk encryption?

Re: I Nearly Lost All of My Data

#25

Post by catmanjan » Sat, Jan 19, 2019, 11:01 PM UTC

jordache wrote: post #21

derekp7 wrote: post #13

The best solution I found as an alternative to an off-site copy is to set up a USB drive and something like a Raspberry Pi in my car. When I pull into the garage, the house server senses it and syncs new data to it. This can be supplemented further by having it auto sync to a computer in your office whenever you pull into work. Add in some monitoring so you can get a phone alert if any of the synced copies are more t…

how do you power the usb drive whilst it's syncing in your car, while parked in the garage? What if the drive gets stolen?

He said it syncs when he pulls in so presumably a 12v power supply

Re: I Nearly Lost All of My Data

#26

Post by cheeseprocedure » Sat, Jan 19, 2019, 11:01 PM UTC

A surge from a lightning strike near my home travelled over the cable line to kill a network switch and the WAN port on a firewall. (Strangely, the cable modem was spared.)

Everything was on a decent UPS... but I’d completely forgotten the cable line.

Re: I Nearly Lost All of My Data

#27

Post by netsharc » Sat, Jan 19, 2019, 11:04 PM UTC

gtirloni wrote: post #24

derekp7 wrote: post #13

The best solution I found as an alternative to an off-site copy is to set up a USB drive and something like a Raspberry Pi in my car. When I pull into the garage, the house server senses it and syncs new data to it. This can be supplemented further by having it auto sync to a computer in your office whenever you pull into work. Add in some monitoring so you can get a phone alert if any of the synced copies are more t…

Does the RaspberryPi have enough CPU for disk encryption?

Not OP, it'd be neat if it just uses FTP (or rsync) to transmit data which is already encrypted from the uploading-side...

Re: I Nearly Lost All of My Data

#28

Post by setquk » Sat, Jan 19, 2019, 11:07 PM UTC

Many years ago I watched someone stuff the DVDRW (remember them?) which contained all his stuff into a work PC’s pioneer slot loader drive to get some music off it. We stood there and the drive went bzzt, clang, bzzt, clang then sped up way faster than it was supposed to go. This was followed by a large bang and bits of DVDRW coming flying out and then a crunching noise.

From this I learned about single points of failure.

Edit: also in the decade and a half since I learned that you should never trust a magic box, magic piece of software or magic container file system for backups. A plain file system you can just copy your shit back from is the closest thing to a guarantee. Also it’s cheaper to curate your data carefully than end up with 4TiB of crap you’re too scared to deal with on your hands.

Re: I Nearly Lost All of My Data

#29

Post by mholt » Sat, Jan 19, 2019, 11:08 PM UTC

These kinds of scenarios are why I built Relica [1]. It backs up to local disks, network drives, remote computers (on LAN or anywhere with a public IP address), your own cloud storage, or our own special formula we call the Relica Cloud: one upload, five independent cloud providers -- replicated in real-time.

And restores can use the open-source tool restic [2], so you don't have to be locked into Relica for accessing your data.

We're working on the ability to do byte-for-byte copies of a repository to other destinations to make data even harder to lose in these kinds of disaster scenarios, as well as a new UI to make it more pleasant and powerful.

Anyway, our goal is help make robust backup strategies like what this guy needs really, really painless, because I'm as paranoid as he should have been about losing data.

[1]: https://relicabackup.com

[2]: https://github.com/restic/restic

Re: I Nearly Lost All of My Data

#30

Post by leowoo91 » Sat, Jan 19, 2019, 11:10 PM UTC

Filligree wrote: post #5

bsder wrote: post #2

Z ... F ... S. And at least Raid-Z. Raid-5 and Raid-6 are now at probability of failure levels that your rebuild is likely to throw an error.

ZFS is great, but won't help if there's a power surge that fries all the disks at once. To prevent that you'll want to store snapshots remotely somewhere.

True story: I've seen Raid5 not recovering after a simple power loss. Software was working but couldn't build it up. Fortunately, we didn't have to dig further than wiping it.