Backups Aren't Simple • Hacker News Board

Re: Backups Aren't Simple

#181

Post by pariainterpares » Thu, Sep 17, 2026, 12:45 PM UTC

wallst07 wrote: post #151

baxtr wrote: post #79

Earlier quoted context omitted.

Since we’re talking about it: anyone backing up their iCloud photos? If so, how?

Amazing question with nobody really answering your question 100%. The answer is : it's hard. If you only have a iphone and more photos on your icloud than can fit on your phone, you're kinda screwed if you don't know what you're doing. AFAIK Apple makes it very difficult to get all your icloud photos bulk downloaded, original quality off icloud. Google is no better. It's much easier to start backing up from the begin…

Not a very good answer.

If you have a Mac with sufficient storage, you can just enable iCloud to store photos locally, then export them. That could miss some data (I think descriptions you set for a photo were not exported the last time I checked), but it is most of the data.

Re: Backups Aren't Simple

#182

Post by dml2135 » Thu, Sep 17, 2026, 12:57 PM UTC

baxtr wrote: post #79

publlus_enigma wrote: post #20

There are four times in my life I have suffered regrettable data loss incidents. The first was when the telephone pole outside our house was struck directly by lightning. Not only was it the loudest thing I have ever heard, the current surged through the telephone line, into the internal fax modem, and fries everything within its vicinity. I was 10. I did have backuos, but only only floppy and they didn't cover every…

Since we’re talking about it: anyone backing up their iCloud photos? If so, how?

Time Machine on my Mac

Re: Backups Aren't Simple

#183

Post by ChrisRR » Thu, Sep 17, 2026, 1:15 PM UTC

bloomingeek wrote: post #4

"encrypted, chunk-level deduplicated, GFS-rotated, point-in-time archived, cloud, 3-2-1 backup solution" is now my newest password, no commas. (Don't tell anyone!)

Really? It's showing up as ***** for me

Re: Backups Aren't Simple

#184

Post by Oxodao » Thu, Sep 17, 2026, 1:19 PM UTC

cuvert wrote: post #178

Oxodao wrote: post #135

Earlier quoted context omitted.

sync is not backup Also why you sync through LAN only?

I was in a vacation and I couldn't connect to a remote pc. And since then I set up wireguard in all routers and I also changed syncthing LAN only.

Ok, i do use it through their relay but i should do the same

Re: Backups Aren't Simple

#185

Post by PunchyHamster » Thu, Sep 17, 2026, 1:20 PM UTC

LilBytes wrote: post #14

PunchyHamster wrote: post #3

So is apparently making readable site. Software like https://restic.net/ does a good job. Few choices of backend Accepts stuff from pipe too so you can just pipe mysqldump or pg_dumpall without intermediate files Very decent options for checking repo integrity, personally (well, at work too, we use it on few hundred user machines and servers) I also added "roll a dice for backup and try to restore it" test script to…

I'm using Restic & Backrest ( https://garethgeorge.github.io/backrest/ ) everywhere I can, they're both exceptional OSS. IMHO, the easiest way to perform a restore test is to use your production backups to refresh a lesser environment on a weekly cadence. Naturally this has to be appropriate to the type of data you're restoring, e.g., our E2E (end to end) testing environment has been built to the same risk tolerances…

We actually do use some of our backups in anonymize -> test environment pipeline so it gets dual use on top of just testing restores.

We also use S3 versioning in case something gets the keys and just erases it. Then it's replicated to the cloud S3-compatible store to get the offsite location.

Only feature I really miss is "turn this restore into .tar", both for testing restores and for convenience when we get requests of "we need this directory from 4 weeks ago"

Re: Backups Aren't Simple

#186

Post by VGHN7XDuOXPAzol » Thu, Sep 17, 2026, 1:40 PM UTC

PunchyHamster wrote: post #3

So is apparently making readable site. Software like https://restic.net/ does a good job. Few choices of backend Accepts stuff from pipe too so you can just pipe mysqldump or pg_dumpall without intermediate files Very decent options for checking repo integrity, personally (well, at work too, we use it on few hundred user machines and servers) I also added "roll a dice for backup and try to restore it" test script to…

To help others disambiguate choices a bit:

Kopia does not let you pipe into it without intermediate files, or at least not in a friendly way I could figure out.

Kopia seems quite designed around the backup server being the place where the config is set. IMO it's harder to configure if you have your own declarative config tooling (like Ansible, NixOS, etc)

The nicest way I found to ignore files declaratively is to configure a ignore-file pattern in the repository policy and then declaratively create `.kopiaignore` files.

The Kopia server is convenient if you have a server at home and want to back up laptops to it, with them being in a shared deduplicated repository without having to trust the clients not to get pwned and wipe out the backups. However I'm not aware of a nice way to then transfer those backups to aggregate them into a bigger repository. (So I just rsync them off-site and don't worry about getting deduplication with other backups) Also learnt the hard way that you can't set a bandwidth limit when backing up to the kopia server, which can tank wifi performance on laptops during the backup.

I am thinking of switching to Restic as it seems like it is more suitable for people who want a tool at the complexity level of tar/git CLI with declarative config, rather than something imperatively configured.

Re: Backups Aren't Simple

#187

Post by cranberryjoe » Thu, Sep 17, 2026, 1:40 PM UTC

Yes, yes they are simple. Btrfs and zfs have incredible snapshot and replicate

Re: Backups Aren't Simple

#188

Post by suncore » Thu, Sep 17, 2026, 2:08 PM UTC

OroPla wrote: post #164

GTP wrote: post #145

Earlier quoted context omitted.

If you have lots of data, I would suggest using a filesystem designed for this, like ZFS or BTRFS. You would still have to spend in storage though, as ultimately protecting from bit rot requires redundancy.

I do use btrfs, but I don't really understand how to use it to protect against bit rot. I tried to look into it at some point, but got overwhelmed. Did invest in two backup copies, one of them off site, though. However, looking at prices, I can see one of those backups just being dismantled to be used as more storage.

Zfs and btrfs will avoid corruption caused below the file system layer, not something like malware or misuse from a tired sysadmin. I run a tool called integrit that creates checksums and lets me detect non-wanted changes. I only run it on important stuff that does not change too much like photos and videos. Not fun to have to regularly check the log but it works fairly well. This could maybe be extended with automatic checking of the logs.

Re: Backups Aren't Simple

#189

Post by publlus_enigma » Thu, Sep 17, 2026, 2:23 PM UTC

dsego wrote: post #63

mrheosuper wrote: post #41

Earlier quoted context omitted.

Sadly after 22 years, it's still considered a "high-end" feature. Even on $1600 Fujifilm x100vi, there is still single SD slot.

I also have the same fuji. Digital cameras are still in the stone ages. I think Tony Northrup covered it well. Nothing has changed since digital cameras came to market, the workflow is still the same, copy files from a card to PC. They aren't connected to the internet, they don't have automatic syncing to cloud for backups or sharing, fuji doesn't even have a gps chip built in to store the location (only works if con…

I actually prefer this workflow, although GPS would be a nice addition.

Re: Backups Aren't Simple

#190

Post by Kinrany » Thu, Sep 17, 2026, 2:27 PM UTC

There should be a standard for backups, one that makes it possible for a (potentially distributed) system to talk to a backup system and to save or restore a snapshot.