Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Backups Aren't Simple

filipovski.net

Post reply on HN
Print view View original

111–120 of 203 posts

Re: Backups Aren't Simple

#111

Post by unanimous »

baxtr wrote: post #79
publlus_enigma wrote: post #20

There are four times in my life I have suffered regrettable data loss incidents. The first was when the telephone pole outside our house was struck directly by lightning. Not only was it the loudest thing I have ever heard, the current surged through the telephone line, into the internal fax modem, and fries everything within its vicinity. I was 10. I did have backuos, but only only floppy and they didn't cover every…

Since we’re talking about it: anyone backing up their iCloud photos? If so, how?

When I was using Ubuntu, I would plug my phone into my laptop and run this small script: https://gist.github.com/wheelercj/8cce112fc8ee55fd4858847a52...

It's mainly just rsync. When the phone was plugged in, the photos were accessible at a path like "/run/user/1000/gvfs/gphoto2:host=Apple_Inc._iPhone_abc123/".

Now I use Time Machine, but I still need to check whether it includes iCloud photos.

Re: Backups Aren't Simple

#112

Post by reddalo »

baxtr wrote: post #79
publlus_enigma wrote: post #20

There are four times in my life I have suffered regrettable data loss incidents. The first was when the telephone pole outside our house was struck directly by lightning. Not only was it the loudest thing I have ever heard, the current surged through the telephone line, into the internal fax modem, and fries everything within its vicinity. I was 10. I did have backuos, but only only floppy and they didn't cover every…

Since we’re talking about it: anyone backing up their iCloud photos? If so, how?

It's not iCloud Photos, in my case it's Google Photos. But I keep a copy of all my photos in a self-hosted instance of Immich [1], which I then back up to a Hetzner Storage Box using Backrest [2].

So I end up with 3 copies of all my photos, on 3 different providers (I mean... 2 different providers plus my homelab disk).

[1] https://immich.app/ [2] https://github.com/garethgeorge/backrest

Re: Backups Aren't Simple

#113

Post by Dylan16807 »

innocent_name wrote: post #17
TheCondor wrote: post #6

Earlier quoted context omitted.

It is a special sensation when you go to your hand rolled backup "solution" to make some simple recoveries and start finding problems. It is a complex mix of not dread but disgust, sick to your stomach, self-anger, and loss. I can only image what that's like with like all of your children's baby pictures being the lost material. Borg and restic are fantastic, it still seems like some sort of dummy proof mode might be…

>Borg ... fantastic I needed to temporarily move off 20TB from my NAS and bought a Hetzner storage box. I wanted to encrypt the data before sending it so i went with the borg ssh mount. Long story short, borg failed a couple of times due to https://github.com/borgbackup/borg/issues/7672 , i resumed it per documentation and got 2gb of silently corrupted data. There was no way to do data integrity checks due to broken…

That issue happens when the repo is corrupted? That's not the cause of the failure then.

Also you must have been unaware of Backblaze B2? It doesn't have a proprietary client, and it launched 11 years ago only a couple months after attic was forked to create borg.

Re: Backups Aren't Simple

#114

Post by ygouzerh »

flippingheck wrote: post #96

> There are two types of people: those who have suffered a catastrophic loss of data, and those who will. This is an empirical claim, but is it grounded in reality? 99% of people don't backup, and that's probably the right choice because the risk is low, they can't meaningfully improve their restoration rate themselves and they don't care enough about their data to classify its loss as catastrophic. When a person hea…

I think this is more a quote for engineers

Re: Backups Aren't Simple

#116

Post by lloydatkinson »

unanimous wrote: post #111
baxtr wrote: post #79

Earlier quoted context omitted.

Since we’re talking about it: anyone backing up their iCloud photos? If so, how?

When I was using Ubuntu, I would plug my phone into my laptop and run this small script: https://gist.github.com/wheelercj/8cce112fc8ee55fd4858847a52... It's mainly just rsync. When the phone was plugged in, the photos were accessible at a path like "/run/user/1000/gvfs/gphoto2:host=Apple_Inc._iPhone_abc123/". Now I use Time Machine, but I still need to check whether it includes iCloud photos.

Does this include all the image metadata and who knows what else?

Re: Backups Aren't Simple

#117

Post by afilipovski »

flippingheck wrote: post #96

> There are two types of people: those who have suffered a catastrophic loss of data, and those who will. This is an empirical claim, but is it grounded in reality? 99% of people don't backup, and that's probably the right choice because the risk is low, they can't meaningfully improve their restoration rate themselves and they don't care enough about their data to classify its loss as catastrophic. When a person hea…

Let's say SSDs have an annualized failure rate of 1%. (The real figures are from 0.5% for more reliable ones, 1.5% for less reliable ones). The annualized failure rate for HDDs is a bit higher, but let's give it 1% anyway.

Over 40 years (even with changing disks, upgrading PCs, etc.), the chances of you ending up with your data intact is 0.99^40=0.67. May not seem too bad, but this is just a hardware failure of your main drive we're talking about. When you factor in all the other types of screwups that happen, the annualized failure rate will skyrocket.

So a more conservative version of this quote is that a 20-year old today is very likely to experience data loss over his lifetime.

Thankfully with cloud being ubiquitous at least photos are given one copy in the cloud, but IMO even that is risky. People have different appetites for risk, and that's OK. But over the last 20 years the all of our data has become pretty important. I don't know how many people could stomach the loss of important documents, photos, etc.

Re: Backups Aren't Simple

#119

Post by dirkc »

> “There are two types of people: those who have suffered a catastrophic loss of data, and those who will.”

When I was a teenager, I was the reason for data loss for my dad, twice. Both times it was because I was re-partitioning a hard drive to install linux.

You would think that taught me a lesson about backups, instead it reminds me every now and again to be grateful for an awesome dad and aspire to handle situations with my kid similarly :)

Re: Backups Aren't Simple

#120

Post by Gigachad »

justsomehnguy wrote: post #107
Gigachad wrote: post #105

Earlier quoted context omitted.

This is for backups. The threat here is that random ware uses your backup script to delete/encrypt the backups too. With object lock the api key the script has can only add data. The probability that the object store goes out of business at the exact time your own copy dies is insignificant.

No object lock API would help you if the business where you rent the service is going out of... business. With all your data - and you are without. Or even fancier - just changing the price for the egress.

That only matters if it happens literally at the same time as a failure of your primary storage. If they increase the price you can just close your account and move the backup to another service.

The probability of a hard drive failure or ransomware at the same time as backblaze or aws going out of business is pretty much not worth thinking about.

The probability of an attacker using the api key in your backup script to destroy the backups is far more possible.

Post reply on HN