Ask HN: How do you backup your files without depending on a third party service?

I use git-annex to do pretty much what you're describing. It takes some getting used to, but it's much better than your suggested method of rsyncing to some USB sticks.

Under the hood it's basically doing that, but everything's SHA-256 checksummed so you can check its validity, and all copies of your data also sync metadata telling them who else has copies, it's easy to enforce policies that some content must have >N copies etc.

I have a core file server that has a 9TB RAID-Z array in it. This server uses key-based authentication to connect to all of my other hosts - my desktop, VPS, work systems, etc and uses rsync to copy any changes to a locally maintained copy. It also attempts to back up laptops, if they are left on overnight.

Every night this system also reaches out to a remote system in my office that has an 8TB RAID-5 array. It "pushes" any changes, again using rsync, to this remote host. This time it uses the --link-dest argument to create nightly "snapshots." If a file hasn't changed, it simply gets hardlinked. This backup volume is also encrypted, and only the core file server has the key to decrypt it.

For the remote system, I have a script that runs periodically that purges backups according to my own specification. Right now it keeps unlimited yearly backups, three years of monthly backups, six months of weekly backups, and 14 days of daily backups.

Once a month I also run a script to back up the core file server to an external 8TB hard drive, again with --link-dest, creating snapshots. This hard drive is stored in a fireproof safe. It is also encrypted.

I have various levels of redundancy elsewhere, too. (Yes, RAID/redundancy is not a backup). For example, my desktop system has a RAID1 array in it.

In 2000, I installed Windows ME on my desktop system. It corrupted my hard drive somehow, and I lost all of my work prior to then. In case you can't tell, this scarred me for life, and I have worked hard to make sure it cannot happen again.

I'll add, every single storage device that I use is encrypted. Sometimes it's just with a key stored on a separate boot disk (usually an SSD). The rationale being if I ever have to RMA a disk, I don't want the manufacturer to be able to see any of my data.

On another tangent, the remote system mentioned above is actually a rather old GX260 with a SATA expansion card and 3 4TB disks. The boot disk is an ancient EIDE 40GB drive. I keep it around mostly for entertainment at this point. Here's why:

  9 Power_On_Hours          0x0012   001   001   001    Old_age   Always   FAILING_NOW 123756

That's right, the drive has been spinning for a little over 14 years.

Unfortunately I've recently hit a problem where e2fsck runs out of memory (it's a 32-bit CPU, so only 2GiB per process) checking the file system. I tried setting a scratch file, but that bombed for some reason too. So, I may finally have to upgrade the thing to a 64-bit system.

Oh well.

Another thought that might make it even simpler (for the user) is to have the device download the data at a throttled rate so it could be done during the workday without impacting the network too much. This way it would be unnecessary to leave it overnight or perhaps even take it out of the purse, backpack, or briefcase. It would still be necessary for the user to remember to recharge it and maybe switch it out or rotate it with other devices.

RClone ( https://rclone.org/ ) synchronizes (similar to rcopy) directories with a multitude of providers and allows for client-side encryption. I use a combination of Google Cloud Storage (for really important files and pictures) with the "crypt" extension and an USB HDD Hub (4 disks) with Raid 5 hooked to a Raspberry PI for less important data. The backup scripts start on boot. It's a charm. Oh and rclone is written in Go so it works on all platforms, including Windows, macOS, Linux (and also ARM).

I mentally added a comma between “on-site NAS” and “Google Drive”. Might be wrong.