Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: How do you backup your files without depending on a third party service?

news.ycombinator.com

Post reply on HN
Print view View original

61–70 of 112 posts

Re: Ask HN: How do you backup your files without depending on a third party service?

#61

Post by koolba »

Jipazgqmnm wrote: post #23
koolba wrote: post #12

If you’re tech savvy and want the easy answer then check out Tarsnap[1]. If you’re tech savvy and want to save a few bucks by doing it yourself, check out borg[2]. You’ll need to set up an external server for it but at scale (TB) it’s an order of magnitude cheaper than Tarsnap. Regardless of off site storage I’d suggest also buying two large USB drives and formatting them with LUKS. Keep them offline and only sync th…

You’ll need to set up an external borg-server ONCE. From there you can additionally sync it to where ever you want using the method you want. E.g. Google cloud. As long as it's encrypted, you don't need to bother about the hosters activities. But forget the USB-sticks. E.g.: Borg to local RAID every 2 hours, sync RAID with cloud every 24 hours.

> But forget the USB-sticks.

> E.g.: Borg to local RAID every 2 hours, sync RAID with cloud every 24 hours.

The problem with a fully online (non-append only) backup system is that if everything is working properly then an accidental 'rm -rf /' will propagate throughout all your supposed backups and destroy them as well. The USB drives are for that type of "Oh shit!" moment as a (possibly stale but better than nothing) backup of last resort

Re: Ask HN: How do you backup your files without depending on a third party service?

#62

Post by toyg »

sbjs wrote: post #25
toyg wrote: post #11

The issue with "simple" backups is that, should your house/office burn down, you lose everything, including the backups. Any alternative (like bringing the usb sticks to a safety box etc) is much more cumbersome than standard cloud-based solutions. I have a SpiderOak subscription for 2TB that is private enough for my needs, and that's where I backup pretty much any machine i have. I also keep a low-power Linux server…

Not if I carry the USB sticks in my pocket at all times!

That's a recipe for losing data even more frequently...

Re: Ask HN: How do you backup your files without depending on a third party service?

#63

Post by joveian »

I've done exactly that to a couple of transcend usb sticks and managed to kill one when deleting some files while the rsync was running (to free up enough space for the rsync to finish). I think the controllers on the sticks are just not as good as SSDs. At least it was an obvious problem (drive completely unresponsive after that), but I wouldn't make that your only copy.

The main thing I have been doing is using old hard drives, each in an inexpensive case[0] that can easily be attached to a usb to SATA/IDE adaptor[1] (there are others to consider if you don't have IDE drives, but that one seems to work well and not all of them do; for SATA only look at UASP). You might think that old drives are more likely to die randomly, but I haven't had that happen once yet (good to keep a couple of copies of stuff you care about).

[0] https://www.fasttech.com/p/1645000 [1] https://www.fasttech.com/p/8292100

I sometimes use rsync but also have an archive script that a) runs mtree with sha512 per file and leaves the file in the directory being archived, b) creates a .tar.xz with SHA256 checksum in the xz (except for already compressed stuff, then I either rsync or use gzip with a seperate SHA256 sum), and c) encrypts most things via scrypt -t 5 before hitting unencrypted external disk (ext2 with limited features for maximum portability). Not exactly push a button level convenience but it works ok for me.

If you just use basic rsync you can end up copying corrupted files from your system over the backup before noticing that they are corrupted. I think there are various ways to avoid this, so far I just have multiple copies that I update at different times but there are likely better ways.

Tarsnap seems like a good option for stuff you wouldn't want to loose in a fire, but I haven't used it so far.

Re: Ask HN: How do you backup your files without depending on a third party service?

#64

Post by ValentineC »

old-gregg wrote: post #7

Synology or QNAP NAS. They collect backups from all computer in the house, encrypt them and ship the encrypted blobs to AWS Glacier. Synology in particular is a fantastic platform which does way more than a traditional NAS. It's quite polished personal cloud platform (way more so than FreeNAS) and includes hundreds of apps, even the backups into the cloud can be done in different ways. I also place a premium on the f…

To add on: Synology largely repackages open protocols into a fairly user-friendly package. Connections can be made over SMB/AFP/NFS, and their Synology Hybrid RAID for utilising different-sized disks is basically mdadm + LVM.

I think it helps if anyone's concerned with avoiding vendor lock-in.

Re: Ask HN: How do you backup your files without depending on a third party service?

#65

Post by sbjs »

wristmittens wrote: post #55

A lot of the replies either rely on third party services (albeit encrypted) or forget about the third part of the holy backup trinity of having something offsite. It sounds like you're thinking of local backup while you're traveling, which is great. The portability of USB sticks seems perfect for your use case, although I think a 1TB stick is still a stretch, maybe look into a small self-powered external SSD drive. I…

The common wisdom always says have an off-site backup. But what if the backup is small enough to fit in your pocket? I always have a Swiss Army knife on me anyway so if I have a backup thumb drive that’s the same size, it should fit just fine, and be as safe as my house keys. I really think we’re in a new age of self sufficiency because of advances in technology and physical storage size.

Re: Ask HN: How do you backup your files without depending on a third party service?

#66

Post by davchana »

A 3TB main hard disk HD01. It has everything of mine from 1998, photos, docs, code, media (self-generated) books everything. All things sorted by type in folders (Media, Own Data, Books, Video, Cloud(Dropbox, Yandex, Mega, Drive)).

An exact mirror of above HD01, labelled aa HD02. Mirror updated every 20-30 days.

An exact mirror of above HD01 in cloud(Yandex & Mega). Mirror updated every 20-30 days. This does not get downloaded to PC back to cause recursion (Cloud on Hard Disk & Hard Disk to cloud)

Few working folders on PC, worked upon, work in progress, mirrored back to their original place on HD01 every week or 10 days.

Stuff of two phones & one tablet gets synced to Dropbox, Yandex & Mega as daily or hourly via foldersync app (Whatsapp backup is synced daily whereas Camera folder as soon as possible). This gets downloaded on PC boa Dropbox or other windows versions. This whole cloud gets synced to HD01 every week or 10 days. Also PC's Chrome's download folder also get mirrored to Cloud very often.

Stuff on gitlab, github & tumblr gets downloaded 1st of every month to Google Drive via Google Apps Script with an email report of log. This also gets downloaded to PC via Drive windows app. This Drive folder is synced to HD01 in Cloud folder.

All sync profiles on PC are setup in FreeFileSync with Drive Labels, Exclusions, Inclusions, Filters & limits.

HD02 is at different house, brought in only for updating monthly.

Re: Ask HN: How do you backup your files without depending on a third party service?

#68

Post by iooi »

old-gregg wrote: post #7

Synology or QNAP NAS. They collect backups from all computer in the house, encrypt them and ship the encrypted blobs to AWS Glacier. Synology in particular is a fantastic platform which does way more than a traditional NAS. It's quite polished personal cloud platform (way more so than FreeNAS) and includes hundreds of apps, even the backups into the cloud can be done in different ways. I also place a premium on the f…

I would stay away from QNAP, their quality is terrible. I had a QNAP QVR and it just reeked of insecurity.

My friend has one of the smaller Synology boxes with 20TB and it is pretty nice. You can think of it as the Apple of the NAS systems out there, everything just works.

However, if you like to tinker with your NAS a bit more, FreeNAS is excellent. It's pretty comforting running zfs over standard RAID, and you have a FreeBSD system where you can run anything you want in native jails.

Re: Ask HN: How do you backup your files without depending on a third party service?

#69

Post by eltoozero »

hb3b wrote: post #51
old-gregg wrote: post #7

Synology or QNAP NAS. They collect backups from all computer in the house, encrypt them and ship the encrypted blobs to AWS Glacier. Synology in particular is a fantastic platform which does way more than a traditional NAS. It's quite polished personal cloud platform (way more so than FreeNAS) and includes hundreds of apps, even the backups into the cloud can be done in different ways. I also place a premium on the f…

Synology is great however I'd like to caution anyone considering their enterprise gear that they do not offer traditional 24/7 support. Regardless of which unit you purchase (in our case we purchased the most expensive RU model), all support tickets are triaged and prioritized equally, and there is no phone support (unless you scream loud).

Get your reseller involved, and if you don't have one; get one. They do local seminars, you can just attend to become one yourself.

Re: Ask HN: How do you backup your files without depending on a third party service?

#70

Post by eltoozero »

Go by the 3-2-1 rule: 3 backups, 2 different formats, one offsite.

Similar to the military mantra: three is two, two is one, one is none. (Meaning your "spare" should already be considered in production or broken, so you need a "spare spare" to really have a "spare").

Time Machine your Macs to a drive and also perform an image backup to a second drive, keep them both in separate places and implement a rotating image or Time Machine backup.

On your PCs File History is probably adequate, and you can image the drive with something free like Acronis.

Linux there are a hundred ways, but the o'reilly 'crocodile' book (backup and recovery) has many good solutions.

Also as a tip you can get solid LTO5 mechanisms for a good price these days on eBay and they're pretty robust and hold a good chunk of bits.

Synology NAS are good machines.

Post reply on HN