Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What are your personal backup / sync strategies?

news.ycombinator.com

Post reply on HN
Print view View original

1–10 of 21 posts

Ask HN: What are your personal backup / sync strategies?

#1

Post by jondot »

Hi all, Due to recent data loss, my need for a solid foolproof way to backup has resurrected itself.

I have:

* Google Drive (100GB tier 2$/mo)

* Amazon Cloud Drive (Unlimited/any file tier)

EDIT: I also have my home Synology NAS

My workflow/strategy that failed:

* GDrive syncs only the root files and not all folders. I keep there notes mainly.

* I once in a while upload RAW image files from my Nikon DSLR to the Amazon Cloud Drive.

* I once in a month connect my external HDD to do Time Machine backup.

* Most of my personal projects are in Github and Bitbucket and I commit and push in a sane way.

* I have an 'experiments' and 'projects' folders in which I do most my work. 'Experiments' is a project that is incubating - it doesn't have a git repo or presence. A project is something that's fully maintained and has a Git repo, and I've decided to invest myself in officially.

During the HDD failure I've lost around 2-3 weeks of hard work, which i spent my vacation re-doing. Most of the work was POC's that I didn't consider a "project" and commits I hadn't pushed yet (no special reason). I worked on many projects in parallel.

My thoughts right now:

* Everything should live within a Git repo? (but then I need to force myself to push all the time)

* Everything should live on Google Drive? My entire home folder? anyone doing this?

* Then again - what should be on Amazon Cloud Drive? it is unlimited..

Please share your backup strategy if you can :)

Thanks!

Re: Ask HN: What are your personal backup / sync strategies?

#2

Post by smt88 »

My code syncs to Dropbox. All of my projects (even experiments) are committed to private Github repos. I have two machines, so Dropbox essentially creates a physical backup on my secondary machine, which is nice.

I also have scheduled backups to an external drive, which enables file history as well.

So for me to lose my files, my main machine, my secondary machine, my external drive, Github, and Dropbox would all have to lose my data before I could get to the backups at any one of those locations (and all of them have incremental backups enabled, so I keep my entire history of changes).

Edit: Most of my clients and several of my organizations use Dropbox to send secret stuff, so Dropbox is already a single point-of-failure for my security. However, I am planning to move to something self-hosted and encrypted at some point. I'd love to put encrypted snapshots in Glacier from time to time and then have my main backups somewhere more accessible.

Re: Ask HN: What are your personal backup / sync strategies?

#3

Post by jondot »

smt88 wrote: post #2

My code syncs to Dropbox. All of my projects (even experiments) are committed to private Github repos. I have two machines, so Dropbox essentially creates a physical backup on my secondary machine, which is nice. I also have scheduled backups to an external drive, which enables file history as well. So for me to lose my files, my main machine, my secondary machine, my external drive, Github, and Dropbox would all hav…

Thanks - so your code lives both on Dropbox and Github? Also - you probably ln -s'd your workspace folder to your Dropbox folder?

Re: Ask HN: What are your personal backup / sync strategies?

#5

Post by adrianhoward »

For our "internal" stuff:

* Monthly whole drive snapshots to an external HD

* Daily Time Machine backups to an external HD

* Everything backed up continually with CrashPlan

* All work stuff synced continuously in DropBox

* Monthly restore-a-random-file test to sanity check

Re: Ask HN: What are your personal backup / sync strategies?

#6

Post by CamTin »

I can't recommend Tarsnap (https://www.tarsnap.com/) enough for personal backups for hacker-types. Everything is encrypted client-side, so don't lose you key or thee is literally nothing you can do to get at your data again. It's also deduplicated and compressed so you can do snapshots often (on my machines I have a cron that runs every six hours) without going broke since (similar to git) you're only storing changes.

Re: Ask HN: What are your personal backup / sync strategies?

#7

Post by walterbell »

CamTin wrote: post #6

I can't recommend Tarsnap ( https://www.tarsnap.com/ ) enough for personal backups for hacker-types. Everything is encrypted client-side, so don't lose you key or thee is literally nothing you can do to get at your data again. It's also deduplicated and compressed so you can do snapshots often (on my machines I have a cron that runs every six hours) without going broke since (similar to git) you're only storing chang…

How does dedupe work with client-side encryption -- are the snapshots and diffs done on the client before encryption and upload?

Re: Ask HN: What are your personal backup / sync strategies?

#8

Post by CamTin »

walterbell wrote: post #7
CamTin wrote: post #6

I can't recommend Tarsnap ( https://www.tarsnap.com/ ) enough for personal backups for hacker-types. Everything is encrypted client-side, so don't lose you key or thee is literally nothing you can do to get at your data again. It's also deduplicated and compressed so you can do snapshots often (on my machines I have a cron that runs every six hours) without going broke since (similar to git) you're only storing chang…

How does dedupe work with client-side encryption -- are the snapshots and diffs done on the client before encryption and upload?

Yes, exactly. The client source is available too if you like, though it's not really free software since the license still only allows you to use it with the official Tarsnap service.

Re: Ask HN: What are your personal backup / sync strategies?

#9

Post by walterbell »

CamTin wrote: post #8
walterbell wrote: post #7

Earlier quoted context omitted.

How does dedupe work with client-side encryption -- are the snapshots and diffs done on the client before encryption and upload?

Yes, exactly. The client source is available too if you like, though it's not really free software since the license still only allows you to use it with the official Tarsnap service.

Sounds useful, I need to look for FOSS software that can do the same for local storage targets.

Re: Ask HN: What are your personal backup / sync strategies?

#10

Post by walterbell »

adrianhoward wrote: post #5

For our "internal" stuff: * Monthly whole drive snapshots to an external HD * Daily Time Machine backups to an external HD * Everything backed up continually with CrashPlan * All work stuff synced continuously in DropBox * Monthly restore-a-random-file test to sanity check

Are your whole drive snapshots deduplicated?
Post reply on HN