Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Backups Aren't Simple

filipovski.net

Post reply on HN
Print view View original

141–150 of 211 posts

Re: Backups Aren't Simple

#141

Post by unanimous »

lloydatkinson wrote: post #116
unanimous wrote: post #111

Earlier quoted context omitted.

When I was using Ubuntu, I would plug my phone into my laptop and run this small script: https://gist.github.com/wheelercj/8cce112fc8ee55fd4858847a52... It's mainly just rsync. When the phone was plugged in, the photos were accessible at a path like "/run/user/1000/gvfs/gphoto2:host=Apple_Inc._iPhone_abc123/". Now I use Time Machine, but I still need to check whether it includes iCloud photos.

Does this include all the image metadata and who knows what else?

While I haven't tested for that myself, the script should keep pretty much everything including image metadata. I just made a small change so that it also keeps extended attributes in case any image-related metadata is there.

The script currently requires changing one or two folder paths at the top (more details in the script).

Re: Backups Aren't Simple

#142

Post by OroPla »

Bit rot was mentioned in the article, but then no solution was offered. I also haven't found a good solution, unfortunately. If you only have a couple of Terabytes you could create check sums of everything and then verify regularly, but I feel this approach doesn't scale very well if you have lots of data.

Re: Backups Aren't Simple

#144

Post by Helmut10001 »

zenoprax wrote: post #125
Helmut10001 wrote: post #83

I really like ZFS snapshots with offsite pull-mode sync using Jim Salter's sanoid/syncoid [1]. ZFS is the base for all OS/filesystems on top of it. If you have a good system for organizing ZFS datasets, and separating ephemeral from persistent data (e.g. [2]), then this is 90% of the backup requirements already fullfilled. [1]: https://github.com/jimsalterjrs/sanoid [2]: https://du.nkel.dev/blog/2026-05-16_rootless_d…

Thanks for the second link - looks interesting. I have an old and loud 16-bay server that boots every 7 days or so if no one is home, `syncoid there here`, and shuts off. I'll get pinged by uptimerobot once in a while if it's overdue and I'll get a notification if a pool is unhealthy or reaching capacity (`sanoid --health` I think). Otherwise I forget I even have it set up. Automated cold backups are great peace-of-m…

Exactly! My offsite Backup box boots using a Shelly Plug S set on timer; it doesn't need to unlock the enrypted ZFS datasets because I use zfs sync snapshots in raw mode (-R). After the zfs scrupt, the box sends an email to me and shuts down. Next week: Repeat.

Re: Backups Aren't Simple

#145

Post by GTP »

OroPla wrote: post #142

Bit rot was mentioned in the article, but then no solution was offered. I also haven't found a good solution, unfortunately. If you only have a couple of Terabytes you could create check sums of everything and then verify regularly, but I feel this approach doesn't scale very well if you have lots of data.

If you have lots of data, I would suggest using a filesystem designed for this, like ZFS or BTRFS. You would still have to spend in storage though, as ultimately protecting from bit rot requires redundancy.

Re: Backups Aren't Simple

#146

Post by b3lvedere »

bloomingeek wrote: post #4

"encrypted, chunk-level deduplicated, GFS-rotated, point-in-time archived, cloud, 3-2-1 backup solution" is now my newest password, no commas. (Don't tell anyone!)

Person entering coffee shop:

"So what is the WiFI password here?"

"You sure would to like to know that eh you effing hipster"

"Does that start with a capital Y?"

"Yes. No spaces."

Re: Backups Aren't Simple

#147

Post by freebsd_lovefes »

Helmut10001 wrote: post #83

I really like ZFS snapshots with offsite pull-mode sync using Jim Salter's sanoid/syncoid [1]. ZFS is the base for all OS/filesystems on top of it. If you have a good system for organizing ZFS datasets, and separating ephemeral from persistent data (e.g. [2]), then this is 90% of the backup requirements already fullfilled. [1]: https://github.com/jimsalterjrs/sanoid [2]: https://du.nkel.dev/blog/2026-05-16_rootless_d…

And FreeBSD comes with native ZFS. MORE, it snapshots your OS before updates/upgrades, so you never ever screw up any upgrade.

Re: Backups Aren't Simple

#148

Post by mesrik »

dirkc wrote: post #119

> “There are two types of people: those who have suffered a catastrophic loss of data, and those who will.” When I was a teenager, I was the reason for data loss for my dad, twice. Both times it was because I was re-partitioning a hard drive to install linux. You would think that taught me a lesson about backups, instead it reminds me every now and again to be grateful for an awesome dad and aspire to handle situatio…

>> “There are two types of people: those who have suffered a catastrophic loss of data, and those who will.”

Also "There are just two types of equipment; those which are already broken and those which will be broken".

I've found a good way to point out anyone thinking right direction not to trust their valuable data in any single entity, product, location, etc.

When I heard it decades ago IIRC 80's Nokia Data Unix courses I went. The distinction made Backups is just 1/3 of the triad Redundancy, Backups, Archives.

1) Redundancy is what you get using RAID, multiple network connectivity etc. ie. good for avoiding single point of failures and avoiding loss service or product availability.

2) Backups are for recovering lost data needed returning lost running known state right before fault or some time state before what backup schedule and used rotation cycle can provide.

3) Archives are meant to saving valuable data not expected to change or active use and but there is need saving much longer periods.

Not seeing and understanding usefulness of difference between these seem hard some people I met over decades. Which then led to confusion, not knowing what they are up to with their data and also often later disappointment when they found out that for example backup programs are not great if you expected archiving instead.

Re: Backups Aren't Simple

#149

Post by Dylan16807 »

mrheosuper wrote: post #139
Dylan16807 wrote: post #109

Earlier quoted context omitted.

Sounds like the "or such" wins, but even with bluetooth speeds that means you're backing up 1-5 photos per minute and that helps a lot.

And harm a lot too, you phone can't deep sleep since it has to wake up periodically to process data from bluetooth stack. Combined with slow transfer speed, that gonna eat your battery like nothing.

Playing audio over bluetooth also wakes things periodically, but it uses almost no power. With a similarly competent design photo transfer should be very battery efficient.

Re: Backups Aren't Simple

#150

Post by wallst07 »

dirkc wrote: post #119

> “There are two types of people: those who have suffered a catastrophic loss of data, and those who will.” When I was a teenager, I was the reason for data loss for my dad, twice. Both times it was because I was re-partitioning a hard drive to install linux. You would think that taught me a lesson about backups, instead it reminds me every now and again to be grateful for an awesome dad and aspire to handle situatio…

Reminds me of a story: I was sitting at a restaurant waiting for my food. I casually notice two young children in the next booth playing with a phone, but pay little attention.

Moments later their dad (presumably) comes back and asks for his phone back and sits down with their food. The kids are eating and the dad is fumbling with his phone with odd looks and I can see his frustration growing.

Then out I hear something like "What did you do to my phone?" repeated loudly over and over. "You erased my phone, it's gone."

The small children (guessing) had figured out how to factory reset his phone.

My thought at the time was something like : How many times did I do something as a toddler I don't remember to piss of my dad like that and that he never got mad or brought it up later (Lots of times knowing myself).

Post reply on HN