Tarsnap – Online backups for the truly paranoid • Hacker News Board

Re: Tarsnap – Online backups for the truly paranoid

#71

Post by danohu » Sat, Jan 28, 2017, 5:10 PM UTC

crzwdjk wrote: post #69

danohu wrote: post #68

Earlier quoted context omitted.

How can I download backups into a different directory from where they originated? [I was just trying out tarsnap, go to the restore test section, and was a bit stumped. It seems like "-xC /other/dir" might do it, but the man page is pretty opaque. I'd like to test a restore without the risk of clobbering my data with some misconfigured backup!]

It works just like tar for the most part, so it extracts into whatever the current working directory happens to be. I'd just do (cd /other/dir; tarsnap -x -f whatever-archive).

Thanks, that makes sense

Re: Tarsnap – Online backups for the truly paranoid

#72

Post by cperciva » Sat, Jan 28, 2017, 5:43 PM UTC

mike-cardwell wrote: post #62

forgotpwtomain wrote: post #26

This has appeared on HN a few times.. I'd really like to use tarsnap but it's simply too expensive to be viable (to which I guess the response is: don't use tarsnap for full-backups -- but this makes it practically useless as far as diminishing the complexity of my current backup scheme[0]). [0] http://duplicity.nongnu.org/

I also use duplicity for my backups. I don't use the S3 option, preferring to write to my own storage, but if I were to turn on S3, what exactly does Tarsnap give me beyond what I'm already getting? Other than an additional middleman taking a cut?

Duplicity isn't bad; I often recommend it to people who want something similar to Tarsnap but don't want Tarsnap itself.

The biggest disadvantage to Duplicity is that it has a far more constrained archive management model -- with tarsnap you can delete any archive at any time, but Duplicity works with a "full plus incrementals" model, which means that (a) you can't delete an archive without deleting the archives which "depend" on it, and (b) you'll inevitably need multiple full archives.

Other points which are probably of lower importance to most users: Duplicity's website and documentation is even worse than Tarsnap's; and they rely on GPG, which has a pretty lousy security track record.

Re: Tarsnap – Online backups for the truly paranoid

#73

Post by scottpiper » Sat, Jan 28, 2017, 6:42 PM UTC

cperciva wrote: post #43

scottpiper wrote: post #34

If you're doing backups for your business, I've written on how to properly encrypt backups[1] and how to use Google Compute Engine for backups[2]. I'm working on write-ups for AWS and Azure that should post within the new few weeks. [1] https://summitroute.com/blog/2016/12/25/creating_disaster_re... [2] https://summitroute.com/blog/2016/12/25/using_google_for_bac...

This is certainly one way to do backups. Two things which come to mind on first reading: 1. You're encrypting backups but not authenticating them; someone with access to your archives could trivially truncate an archive or replace one archive with another, and there's a nontrivial chance that they could splice parts of different archives together. 2. Every archive you're creating is stored as a separate full archive;…

These are both accurate.

For 1, I ensure that an attacker can not modify my archives after they've been uploaded by giving the backup service "put" only privileges. This is not possible with GCE from the article unfortunately, as I point out in a warning banner there, but is with AWS that I'll post soon. My use case is primarily to have a backup in the event of a devops mistake, or malicious attacker (ransomware), so I assume if someone has write-access to my archives they would just delete them, so authenticating them isn't as big of a concern, but although this would be a good idea just to ensure the files aren't corrupt in some other way.

For 2, my storage needs currently aren't expensive (100GB archives per day, which means pennies per day for all of them), but eventually I plan on sending just diffs. I also wanted to create and send backups in the simplest possible way to help people get up and running as fast as possible, which meant limiting myself to the "openssl" command and other basic commands. The other, smarter, solutions I'm aware of are either tied to a service (ex. tarsnap) or don't maintain the data as encrypted at the backup location.

Re: Tarsnap – Online backups for the truly paranoid

#74

Post by db48x » Sun, Jan 29, 2017, 6:19 AM UTC

warmwaffles wrote: post #5

As always, picodollar pricing is so confusing.

The price is quoted in GB-months right on the home page. All the picodollar pricing does is eliminate any uncertainty over rounding.

Re: Tarsnap – Online backups for the truly paranoid

#75

Post by caivvoacmh » Fri, Feb 10, 2017, 6:16 PM UTC

cperciva wrote: post #2

Not sure why Tarsnap has bubbled up on HN again, but I'm always happy to answer questions about Tarsnap if there's anything anyone here wants to know.

Hi cperciva can you please explain the following:

1) "Every Tarsnap archive acts like it is completely independent of all other archives"

Lets say I'm backing up `/somedir` that looks like this:

# archive1 - initial backup

  ── somedir
      ├── file1.txt
      ├── file2.txt
      └── file3.txt

# archive 2 - 2nd backup

  ── somedir
      └── file3.txt (modified)

Are you suggesting that if I delete `archive1` and `rm -rf /somedir` that I can restore `/somedir` with `archive2` alone?

If so, how is that possible?

2) Does tarsnap stream the backup data to the tarsnap service or does it create the encrypted archive and only after this upload it?

For instance, if I've got 100 GB to backup (first time), do I need 100 GB of free space for tarsnap to work?

Re: Tarsnap – Online backups for the truly paranoid

#76

Post by cperciva » Fri, Feb 10, 2017, 7:12 PM UTC

caivvoacmh wrote: post #75

cperciva wrote: post #2

Not sure why Tarsnap has bubbled up on HN again, but I'm always happy to answer questions about Tarsnap if there's anything anyone here wants to know.

Hi cperciva can you please explain the following: 1) "Every Tarsnap archive acts like it is completely independent of all other archives" Lets say I'm backing up `/somedir` that looks like this: # archive1 - initial backup ── somedir ├── file1.txt ├── file2.txt └── file3.txt # archive 2 - 2nd backup ── somedir └── file3.txt (modified) Are you suggesting that if I delete `archive1` and `rm -rf /somedir` that I can res…

Are you suggesting that if I delete `archive1` and `rm -rf /somedir` that I can restore `/somedir` with `archive2` alone?

If archive2 only contains somedir/file3.txt, then if you extract archive2 you'll get somedir/file3.txt but not file1.txt or file2.txt.

But if you create archive2 containing all three files, tarsnap will recognize the duplicate data in file1.txt and file2.txt and not re-upload it; but when you extract archive2 you'll get all three blocks. Tarsnap reference-counts blocks of data so that when you delete archive1 it only deletes the data which is not used by any of the remaining archives.

I often tell new Tarsnap users that they should start by forgetting everything they know about incremental backups. Tell Tarsnap what data you want to have in an archive, and let it do the work of figuring out what's new.

Does tarsnap stream the backup data to the tarsnap service or does it create the encrypted archive and only after this upload it?

Tarsnap uploads data as it collects it. Tarsnap needs a small amount of disk space to keep track of which blocks have been uploaded previously, but it's less than 1% of the size of the data archived.