Live data from Hacker News

I Nearly Lost All of My Data

kevq.uk

141–150 of 262 posts

Re: I Nearly Lost All of My Data

#141
post #74

I just don't understand rolling your own backup drives in 2018 unless you're up to something nefarious. I rest peacefully knowing my data will never be lost by Dropbox, and the immediate backup means I don't have to mess with slow, periodic all-at-once backups.

It was several years ago and I can't find the link now, but Dropbox has previously had a bug that deleted the data in the account and then synced the deleted change back to all the client PCs. It was unrecoverable.

Re: I Nearly Lost All of My Data

#142
post #135

Hard to believe we are 129 comments into this and there are zero mentions of 'borg' in the comment threads ... For those that don't know, borg is a backup utility[1] that has been called the "holy grail of backups"[2]. It takes your plaintext files and directories, chops them into gpg-encrypted chunks with encrypted, random filenames, and will upload (and maintain) them, with an efficient, changes-only update, to any…

restic was mentioned a few times and is a very similar project with very similar goals. They both have encryption as a first-class concept and are snapshot based with content-defined chunking. I'd hope most people would be using these newer generations of backup tools.

Re: I Nearly Lost All of My Data

#143
Something like that happened to me. I had several TB of data on a RAID10 array on an LSI MegaRAID controller. A nearby lightning strike took out the server. And the server manufacturer had gone out of business.

I had backups of the data itself. But I'd been doing lots of data massaging, and didn't have enough storage to keep copies of every step.

Anyway, so I bought a couple new servers. One to replace the dead one, to be setup with SQL Server. And a low-end one that would accept the controller from the old server. I just left the drives in their cage, and jury rigged power and data connections.

And it worked.

Re: I Nearly Lost All of My Data

#144
post #135

Hard to believe we are 129 comments into this and there are zero mentions of 'borg' in the comment threads ... For those that don't know, borg is a backup utility[1] that has been called the "holy grail of backups"[2]. It takes your plaintext files and directories, chops them into gpg-encrypted chunks with encrypted, random filenames, and will upload (and maintain) them, with an efficient, changes-only update, to any…

I feel as though you are missing a "Disclaimer: ...", though I will be looking at borg. :-)

Re: I Nearly Lost All of My Data

#145
post #142
post #135

Hard to believe we are 129 comments into this and there are zero mentions of 'borg' in the comment threads ... For those that don't know, borg is a backup utility[1] that has been called the "holy grail of backups"[2]. It takes your plaintext files and directories, chops them into gpg-encrypted chunks with encrypted, random filenames, and will upload (and maintain) them, with an efficient, changes-only update, to any…

restic was mentioned a few times and is a very similar project with very similar goals. They both have encryption as a first-class concept and are snapshot based with content-defined chunking. I'd hope most people would be using these newer generations of backup tools.

My favorite benefit of restic is that it doesn't require the receiving end to have any special software: you can use restic to backup to S3 or BackBlaze B2 directly.

Re: I Nearly Lost All of My Data

#146
post #125

There really isn't a story here, Kev. You blew a power supply on your Synology. You even said it yourself: "maybe this is a problem with the enclosures and the disks are fine". If you'd put your disks into a replacement Synology unit, you would have been back online - config, data, and all - within a few minutes.

This - I thought of URE as he mentioned Raid5 but the disk size was ok (1tb).

But this scenario? Lost almost all his data? I once closed the terminal window where I spawned gparted from while resizing a partition with it... this was nerverracking

Re: I Nearly Lost All of My Data

#147
post #97
post #95

If you have 4TB of data saved at home you are a hoarder. Why do you have that much stuff. I don't have any data at home that I would miss very much if I lost it.

Family photos and videos can take up a lot of space. Also it sounds like the authors work is backed up to this NAS as well. I wouldn't have that much to backup but some do, depending on their hobbies and work.

Photos, videos... Yeah I used to save that stuff. I realized I never looked at it again. So I just stopped. I literally don't worry about losing data at home. Work is different of course.

Re: I Nearly Lost All of My Data

#148
post #13

The best solution I found as an alternative to an off-site copy is to set up a USB drive and something like a Raspberry Pi in my car. When I pull into the garage, the house server senses it and syncs new data to it. This can be supplemented further by having it auto sync to a computer in your office whenever you pull into work. Add in some monitoring so you can get a phone alert if any of the synced copies are more t…

I use syncthing for syncing when my raspberry pi joins a specific BSSID, sync starts. It's susceptible to macspoofing but I hope my Pi won't join a spoofed network.

I use syncthing to sync (mostly important text stuff) between my two machines and my android phone. Once every day one of the machines uploads the changes via duplicati.

Re: I Nearly Lost All of My Data

#149

Earlier quoted context omitted.

Most encrypted backup solutions are really bad with protecting keys. Fixed ivs are ok for one file. Not ok for possibly millions of small files. Basically exposed your private key along with your backup.

How do you manage millions of keys, if you have millions of small files to be backed up? Would it be ok to have something between 1:all and 1:1?

Sqlite.

Re: I Nearly Lost All of My Data

#150

Kudos to Synology for having a process to recover on a Linux box if the Synology box craps the bed. I was reading this story thinking "this is a good lesson on why you don't buy an expensive SAN, unless you can afford to buy two". But, the other lesson is: Backups. Sounds like backups were shut off 6 months prior. The other lesson is: Monitoring. Backups were going to the USB drive, but it that stopped working at som…

The nice thing about Synology storage stuff is that it's a nice GUI (seriously, they're about the only company I can think of that's doing appliance management right) on top of standard and battle-tested open source tools. This was one of the reasons I was okay with paying their prices. Even if the device completely craps the bed, I'll be able to hook the drives containing absolutely normal LVM/btrfs volumes up to an…

Btrfs is battle tested?

https://btrfs.wiki.kernel.org/index.php/Gotchas

We use a box with 2x2TB mirror zpools. It runs Nas4free off an 128GB SSD. Initially we wanted to use Freenas but ir needs 16GB of RAM, while Nas4free works fine with 8. It also does SMART monitoring sending emails when something is up.

Post reply on HN