Live data from Hacker News

I Nearly Lost All of My Data

kevq.uk

221–230 of 262 posts

Re: I Nearly Lost All of My Data

#221

My first thought is "why don't you keep a backup?" I really don't know what the moral of this story is beyond "don't keep one copy of all your data, especially in the same physical location." Online storage is cheap. Bandwidth is cheap. There's a multitude of solutions in the comments if a consumer solution like Dropbox or Google Drive isn't good enough for you.

> Bandwidth is cheap.

Most ISPs in the US now have a 1TB datacap and charge ridiculous rates for overages.

Re: I Nearly Lost All of My Data

#222

Is it weird that I don't have GBs of my own personal files/data? I get if you have movies/tv shows or whatever, that's different. I just don't know what personal data you could possibly have that would amount to GBs...

As a parent you go through a period of time - when your kid is young - that you take photos and videos of EVERYTHING that they do. Its silly now that i look back. Certainly it is important to capture moments, but to over-document everything is silly...now i take minimal photos/videos, and just be present in the moment. Oh, and my data accumulation has precipitously gone down since i've mellowed out. But, yeah family…

I have 3 kids and still don't have more than a terabyte of data.

Re: I Nearly Lost All of My Data

#223
post #142

Earlier quoted context omitted.

restic was mentioned a few times and is a very similar project with very similar goals. They both have encryption as a first-class concept and are snapshot based with content-defined chunking. I'd hope most people would be using these newer generations of backup tools.

My favorite benefit of restic is that it doesn't require the receiving end to have any special software: you can use restic to backup to S3 or BackBlaze B2 directly.

Same! I’ve been using Duplicity for most of my backups and had been considering switching to Borg, but passed when I found it couldn’t use B2 as a backend.

Now I’m using both Duplicity and Restic, for different backups.

Re: I Nearly Lost All of My Data

#224
post #135

Hard to believe we are 129 comments into this and there are zero mentions of 'borg' in the comment threads ... For those that don't know, borg is a backup utility[1] that has been called the "holy grail of backups"[2]. It takes your plaintext files and directories, chops them into gpg-encrypted chunks with encrypted, random filenames, and will upload (and maintain) them, with an efficient, changes-only update, to any…

The crypto in Borg is bad. Even has its own page in the docs about how bad it really is. The tl;dr is "there is one AES key per repository and we use this with AES-CTR and trust the remote server to not fiddle with the counter". https://borgbackup.readthedocs.io/en/stable/internals/securi... Last time I checked there was still no work being done on improving performance. Multithreading has been on the agenda for year…

I'm not sure that's a fair summary. It's very clearly stated that there is authentication performed after encryption.

The MAC should prevent the remote server changing the counter or any encrypted bits.

Re: I Nearly Lost All of My Data

#227

My first thought is "why don't you keep a backup?" I really don't know what the moral of this story is beyond "don't keep one copy of all your data, especially in the same physical location." Online storage is cheap. Bandwidth is cheap. There's a multitude of solutions in the comments if a consumer solution like Dropbox or Google Drive isn't good enough for you.

> Bandwidth is cheap. Most ISPs in the US now have a 1TB datacap and charge ridiculous rates for overages.

Are you really pushing 1TB/mo, though? That would be a full quarter of OP's lifetime of data. And in those cases where you are, could you not afford business class internet for a bit of a premium?

Re: I Nearly Lost All of My Data

#228

Earlier quoted context omitted.

> Bandwidth is cheap. Most ISPs in the US now have a 1TB datacap and charge ridiculous rates for overages.

Are you really pushing 1TB/mo, though? That would be a full quarter of OP's lifetime of data. And in those cases where you are, could you not afford business class internet for a bit of a premium?

It's on top of your other bandwidth usage though. Family of 4 with Netflix/YouTube/Twitch can accumulate quite a bit. Still, using an online service at least for an important subset is a good idea.

Re: I Nearly Lost All of My Data

#229
post #150

Earlier quoted context omitted.

The nice thing about Synology storage stuff is that it's a nice GUI (seriously, they're about the only company I can think of that's doing appliance management right) on top of standard and battle-tested open source tools. This was one of the reasons I was okay with paying their prices. Even if the device completely craps the bed, I'll be able to hook the drives containing absolutely normal LVM/btrfs volumes up to an…

Btrfs is battle tested? https://btrfs.wiki.kernel.org/index.php/Gotchas We use a box with 2x2TB mirror zpools. It runs Nas4free off an 128GB SSD. Initially we wanted to use Freenas but ir needs 16GB of RAM, while Nas4free works fine with 8. It also does SMART monitoring sending emails when something is up.

On the btrfs Parity Raid issue: Synology uses btrfs on md RAID [1], so I believe it's more stable than the native btrfs implementation.

[1] https://www.synology.com/en-global/knowledgebase/DSM/tutoria...

Re: I Nearly Lost All of My Data

#230

I recently was also thinking about how to organize my data & backups. I still have not really decided. Esp about the software. I collected lots of options here: https://github.com/albertz/wiki/blob/master/backup-software.... At the moment, I really like Perkeep ( https://perkeep.org/ ). But I'm not sure whether this is a solution for everything. On the hardware side, I also have not really decided. I want to build up…

Yes, I was also thinking about perkeep and giving it a spin. I was considering a RAID 1 array for redundancy. Another thing I was considering was M-Disk[1], that can, supposedly, hold information for 1000 years. But I wonder what other people's experience with it is. Alternatively, I understand normal Blu-Ray disks should be able to have data retention of about 40 years, and that sounds decent to me. I was looking in…

> Thinking about it, I think this would be a great idea for a startup. Cheap data storage for long term storage, with competitive prices, the ability to per-pay, user-side encryption and a simple UI that grandma can use to drop photos. It should be able to guarantee that the information you desire to will outlive you.

As a consumer, I wouldn't trust a startup for such needs, because there's a likelihood that the startup would either raise prices, pivot to a different service offering, or shut down entirely.

Google Cloud Platform lets you make a manual prepayment [1], so that's an option worth considering. I know Google gets a lot of flak for shutting down consumer services, but I'm inclined to believe that they wouldn't shut down Nearline/Coldline without a significant amount of notice.

[1] https://cloud.google.com/billing/docs/how-to/manual-payment

Post reply on HN