Backups Aren't Simple • Hacker News Board

Re: Backups Aren't Simple

#201

Post by darkwater » Thu, Sep 17, 2026, 4:14 PM UTC

baxtr wrote: post #79

publlus_enigma wrote: post #20

There are four times in my life I have suffered regrettable data loss incidents. The first was when the telephone pole outside our house was struck directly by lightning. Not only was it the loudest thing I have ever heard, the current surged through the telephone line, into the internal fax modem, and fries everything within its vicinity. I was 10. I did have backuos, but only only floppy and they didn't cover every…

Since we’re talking about it: anyone backing up their iCloud photos? If so, how?

From iPhone to the family's Immich instance on a RAID array and from there to Backblaze

Re: Backups Aren't Simple

#202

Post by graemep » Thu, Sep 17, 2026, 4:26 PM UTC

bnegreve wrote: post #194

graemep wrote: post #89

Earlier quoted context omitted.

rsync can delete remote files with --delete or --delete-after What is missing is recovering accidentally deleted or corrupted data.

> rsync can delete remote files with --delete or --delete-after Yes but I don't use it precisely because that lets me recover files that I have accidentally deleted. I assume it's the intended default behavior of rsync.

What if that happens after your monthly creation of a new backup.

Re: Backups Aren't Simple

#203

Post by branon » Thu, Sep 17, 2026, 4:36 PM UTC

crossroadsguy wrote: post #56

If you don't control your data size, it will end up controlling you and your backup choices, which will eventually lead to many avoidable, disastrous outcomes. Somehow, this gets missed in data storage and backup planning.

Agreed - information always expands to fill the available space, and placing a cap on your available space is an underrated strategy

Re: Backups Aren't Simple

#204

Post by rsync » Thu, Sep 17, 2026, 5:08 PM UTC

zenoprax wrote: post #131

ebrahimh wrote: post #23

I’m setting up 3-2-1-ish backups for my infra of 3 hosts, and definitely leaning towards Restic + Backrest. All my hosts run the same CoreOS setup ( https://github.com/ebrahim37/infra-template ), where container volumes are placed in one central volumes/ folder and that is the only thing I have to backup. I plan to implement it like this: vps1: - restic container with custom sh entrypoint that will backup volumes/ to…

I've been trying to find a solution for this too! I was considering using Rclone but too many things are using SQLite for me to trust rsync. I was also going to go with CoreOS but I'm leaning towards Fedora Cloud now in case I need to manage things a bit more (and "auto updating" is not something I want as that suggests auto rebooting). Your secrets.yaml makes me nervous though - too easy to miss a key and leave some…

Are you aware of sqlite3-rsync ?

It is a tool that was created by the author of sqlite. It does just what you would expected to do.

Re: Backups Aren't Simple

#205

Post by nmacias » Thu, Sep 17, 2026, 5:36 PM UTC

QuiEgo wrote: post #77

Time Machine (to NAS) + iCloud (w/ ADP) + local copy is the lowest friction path I have found.

I had an synology NAS power blip which caused a long term time machine to become completely corrupt, like, i’m not confident its quite so compatible with non apple disk formats

Re: Backups Aren't Simple

#206

Post by Terr_ » Thu, Sep 17, 2026, 5:40 PM UTC

Gigachad wrote: post #124

Terr_ wrote: post #38

Earlier quoted context omitted.

I pretty much instantly wrote off physical media because I know I won't reliably move it to a separate location. If I'm going to have backups, I want ones that are not within fire/flood range of my computer. In terms of preventing "oops" moments, I'm mainly relying on software (restic) for that, where I trust that (A) backups always append data rather than replacing and (B) it's logic works for marking which data to…

You want to make sure the append only is enforced by the hosting provider rather than duplicity. Your backups should be resistant to having an attacker on the system who can run arbitrary commands with duplicity.

Right, defense against internal "oops" accidents looks rather different than defense against purposeful sabotage.

While the local backup service uses an API-key with limited privileges, I need to go back to the bucket/storage-zone settings and see what I can enable, how it would affect total storage, and whether I (with additional privileges) can reliably grab the correct mix of old-blobs to do a restore.

Re: Backups Aren't Simple

#207

Post by bnegreve » Thu, Sep 17, 2026, 6:09 PM UTC

graemep wrote: post #202

bnegreve wrote: post #194

Earlier quoted context omitted.

> rsync can delete remote files with --delete or --delete-after Yes but I don't use it precisely because that lets me recover files that I have accidentally deleted. I assume it's the intended default behavior of rsync.

What if that happens after your monthly creation of a new backup.

I keep the previous backup for one month and then delete it. So eventually the file is lost yes, but this is inevitable if you assume finite amount of storage.

Re: Backups Aren't Simple

#208

Post by voiper1 » Thu, Sep 17, 2026, 6:13 PM UTC

b3lvedere wrote: post #146

bloomingeek wrote: post #4

"encrypted, chunk-level deduplicated, GFS-rotated, point-in-time archived, cloud, 3-2-1 backup solution" is now my newest password, no commas. (Don't tell anyone!)

Person entering coffee shop: "So what is the WiFI password here?" "You sure would to like to know that eh you effing hipster" "Does that start with a capital Y?" "Yes. No spaces."

Rocketjump! The wife password is four words all uppercase.

https://www.youtube.com/watch?v=bLE7zsJk4AI

Re: Backups Aren't Simple

#209

Post by burnt-resistor » Thu, Sep 17, 2026, 6:15 PM UTC

freebsd_lovefes wrote: post #147

Helmut10001 wrote: post #83

I really like ZFS snapshots with offsite pull-mode sync using Jim Salter's sanoid/syncoid [1]. ZFS is the base for all OS/filesystems on top of it. If you have a good system for organizing ZFS datasets, and separating ephemeral from persistent data (e.g. [2]), then this is 90% of the backup requirements already fullfilled. [1]: https://github.com/jimsalterjrs/sanoid [2]: https://du.nkel.dev/blog/2026-05-16_rootless_d…

And FreeBSD comes with native ZFS. MORE, it snapshots your OS before updates/upgrades, so you never ever screw up any upgrade.

It's not native ZFS. They threw the native port away and replaced it with ZoL.

Re: Backups Aren't Simple

#210

Post by OroPla » Thu, Sep 17, 2026, 6:23 PM UTC

GTP wrote: post #197

OroPla wrote: post #164

Earlier quoted context omitted.

I do use btrfs, but I don't really understand how to use it to protect against bit rot. I tried to look into it at some point, but got overwhelmed. Did invest in two backup copies, one of them off site, though. However, looking at prices, I can see one of those backups just being dismantled to be used as more storage.

If you got overwhelmed in trying to understand how to do it, just do it the easy way: create a ZFS pool of two drives of identical size in mirroring. Then, make sure to have regular scrubs (this means ZFS will regularly check all files for corruption, and recover from eventual errors), use your OS' task scheduler (e.g. Systemd or cron) to run this once a month. This should be sufficient to protect you from bit rot.

I've currently got two 10TB, one 14TB and one 18TB drive. I don't think what you describe is as easy for me as you expect. I've got one backup in a different city and one in cold storage.

The sizes are all over the place because I only buy a new drive when I run out of storage and then just get the largest one within a reasonable cost. It has been cost prohibitive to mirror everything and current prices aren't encouraging. Neither is the rate at which HDDs get larger. I was kinda hoping for 100TB drives under $400 by 2025 back in 2014, but that hope has been shattered.