Live data from Hacker News

Tarsnap is cozy

til.andrew-quinn.me

61–70 of 83 posts

Re: Tarsnap is cozy

#61

I really wanted to like Tarsnap and gave it a good hard look for my backup needs. Ultimately my problem was that there's no way for me to gauge how much the service will cost me. Going just by the amount of data in my home dir, it would be cost prohibitive to upload to Tarsnap. The site does assure me that thanks to compression and deduplication, the actual cost will be far less than I might estimate, which is great!…

tarsnap --dry-run --no-default-config --print-stats --humanize-numbers -c /MY/DATADIR will tell you the compressed size of your deduplicated data, which gives you the upload cost and first-month cost. 4GB of files usually works out to 3GB of dedup/compressed archive data for most people, less for people with many similar files.

Thank you very much for the info! I'm glad to have been wrong, and that there is a way to do this.

Re: Tarsnap is cozy

#62
post #49

Earlier quoted context omitted.

+1 for restic Restic + rclone is a very nice combo. Works really well.

Curious why use both? I use restic directly with B2 backblaze, whats rclone doing for you here?

I use restic + rclone to back up to onedrive, where I have 1TB space included with my subscription.

My main backups are on rsync.net, though.

Re: Tarsnap is cozy

#63
post #46

Earlier quoted context omitted.

That’s because restic is not opinionated about where and how you store your backups. Restic provides a nice interface to create the backups, and then lets you choose where you want to store them (and how access to them is managed), be it locally or via SFTP or S3 or many other backends. Any security properties related to S3 are not in the scope of what restic is meant to do. It’s pretty simple to enable versioning an…

I disagree, strongly. Here are the relevant docs: https://restic.readthedocs.io/en/stable/030_preparing_a_new_... I would like to see an explicit discussion of what permissions are needed for what operation. I would also like to see a clearly specified model in which backups can be created in a bucket with less than full permissions and, even after active attack by an agent with those same permissions, one can enumer…

Again, this isn’t at all in the scope of restic’s docs. If you’re using S3 as the storage, it’s on you to understand how S3 works and what permissions are needed, just like it’s on you to understand how your local file system works and file permissions work if you use the local file system as a backend.

If you don’t understand S3 or don’t want to learn, then that’s fine, and you can pay the premium to tarsnap for simplifying it for you. But that’s your choice, not an issue with restic.

If you think differently, have you submitted a PR to restic’s docs to add the information you think should be there?

Re: Tarsnap is cozy

#64
post #47
post #22

Earlier quoted context omitted.

I found restic is a prety cool alternative. (No hosting though, I am sending restic backups to a private server/vps)

Same but with rustic because I found it used significantly less memory. https://rustic.cli.rs/

Is it stable? I've been using restic for a while, and I'm interested in rustic, but I have no idea how stable it is overall. Obviously it's still in beta so I won't use it in prod but curious what others experiences have been like.

Re: Tarsnap is cozy

#65
post #46

Earlier quoted context omitted.

I disagree, strongly. Here are the relevant docs: https://restic.readthedocs.io/en/stable/030_preparing_a_new_... I would like to see an explicit discussion of what permissions are needed for what operation. I would also like to see a clearly specified model in which backups can be created in a bucket with less than full permissions and, even after active attack by an agent with those same permissions, one can enumer…

Again, this isn’t at all in the scope of restic’s docs. If you’re using S3 as the storage, it’s on you to understand how S3 works and what permissions are needed, just like it’s on you to understand how your local file system works and file permissions work if you use the local file system as a backend. If you don’t understand S3 or don’t want to learn, then that’s fine, and you can pay the premium to tarsnap for sim…

Interesting play on the debate- but after the response to restic's original decision to upstream Object Store permissions and features... to the Object Store, along with my attempts to explain S3 to several otherwise reasonably technical people....

I think people are frequently trapped in some way of thinking (not sure exactly) that doesn't allow them to think of storage as anything other than Block based. They repeatedly try to reduce S3 to LBA's, or POSIX permissions (not even modern ACL type permissions), or some other comparison that falls apart quickly.

Best I've come up with is "an object is a burned CD-R." Even that falls apart though

Re: Tarsnap is cozy

#66
post #46

Earlier quoted context omitted.

I disagree, strongly. Here are the relevant docs: https://restic.readthedocs.io/en/stable/030_preparing_a_new_... I would like to see an explicit discussion of what permissions are needed for what operation. I would also like to see a clearly specified model in which backups can be created in a bucket with less than full permissions and, even after active attack by an agent with those same permissions, one can enumer…

Again, this isn’t at all in the scope of restic’s docs. If you’re using S3 as the storage, it’s on you to understand how S3 works and what permissions are needed, just like it’s on you to understand how your local file system works and file permissions work if you use the local file system as a backend. If you don’t understand S3 or don’t want to learn, then that’s fine, and you can pay the premium to tarsnap for sim…

I still completely disagree. It’s on me to understand IAM. It should not be on me to understand the way that restic uses S3 such that I can determine whether I can credibly restore from an S3 bucket after a compromised client gets permission to create objects that didn’t previously exist. Or to create new corrupt versions of existing objects.

For that matter, suppose an attacker modifies an object and replaces it with corrupt or malicious contents, and I detect it, and the previous version still exists. Can the restic client, as written, actually manage the process of restoring it? I do not want to need to patch the client as part of my recovery plan.

(Compare to Tarsnap. By all accounts, if you backup up, your data is there. But there are more than enough reports of people who are unable to usefully recover the data because the client is unbelievably slow. The restore tool needs to do what the user needs it to do in order for the backup to be genuinely useful.)

Re: Tarsnap is cozy

#67

gzip + ccrypt -> thumb drive Also cozy if your data fits. No monthly fee, just the cost of new/recycled thumbies

I love thumb drives, but Tarsnap is cheaper than the expected 10 year lifetime of a fresh and well maintained thumb drive for the kind of data I hold in there by about a factor of 20 (50 cents vs $10). It also doesn't require a UL Class 125 fireproofed safe to survive a house fire, but that's splitting hairs and getting into hobbyist territory.

tarsnap is not cheaper than anything

Re: Tarsnap is cozy

#68
I've been using Tarsnap for almost a decade for a small, but very important, personal data subset.

Tarsnap is very resilient; it doesn't do a lot, but what it does is solid. The mailing list is helpful, and you can reach out to its creators directly for prompt, useful responses if that is something you don't want on the mailing list (where names and email addresses are in the clear; use marc.info to search in it).

But if you are trying to start with Tarsnap, you should note a few things from the beginning:

- If you are looking for a completely (or even almost) frictionless backup experience - this is not it. Also, it doesn't have tons of features - which might be a good thing, but you should know and accept it.

- If you're used to tools like Backblaze, CrashPlan, Restic, or Borg, the limited feature set might frustrate you.

- Knowing this in advance will help you set expectations within its feature set. The doc/man pages are great resources once you actually read it.

- It has some quirks (may or may not be bugs) that require tinkering with your settings, env etc. Getting your hands dirty with sample data first is a great way to know Tarsnap.

- Set up your logs and scripts such that you can know/debug things later.

- Naming of your archives is important.

- You'll need at least two keys: a master key with read, write, and delete access on your archives/Tarnsap storage, and a un-passphrased regular key with only "write" permission for backups. Keep both safe, especially the master key. There's "nuke" as well…

- I used its GUI for the longest time but would absolutely not recommend it. It hides a lot, which might come back to bite you, and is not the most polished tool of all. Its last release was 7 years ago.

OP says:

> … If you use it solely to back up the few megabytes of “crown jewels” data we all have lying around"

and I actually use Tarsnap exclusively for my "crown jewels," which are in the early three digit MBs.

- So, unlike what many say, I do believe it is costly for today's storage/bw prices, especially if your data isn't very compressible. Tarsnap's compression is great, but not magic. However, i doesn't cost an exponential bomb either. Killer de-dupe though.

- You must have a plan for what and how much you want to back up, and the expected growth of that data.

- It is definitely not a "fire and forget" tool (and you should never forget your backups anyway).

I was frustrated with it until I gave up on the GUI, embraced the CLI/cron, reduced the amount of data being backed up and excluded (using copy and delete) some data being stored, and accepted what it can't do. Which is not really great but that's what it is.

Glaring omissions, IMHO: very few maintenance features (the scripts listed are not easy to work with), (almost?) no way of knowing what file changed in a certain archive, slow restores (may matter for a bigger data set), and the lack of an updated, polished GUI tool which I think is very important for personal data backup.

My request to cperciva would be: please consider this - while it's inspired by tar and stays close to it, it's also a cloud backup tool. Treating it a bit more like a modern cloud backup tool could be useful. Just my two cents.

Re: Tarsnap is cozy

#69
post #47

Earlier quoted context omitted.

Same but with rustic because I found it used significantly less memory. https://rustic.cli.rs/

Is it stable? I've been using restic for a while, and I'm interested in rustic, but I have no idea how stable it is overall. Obviously it's still in beta so I won't use it in prod but curious what others experiences have been like.

You might want to check this out https://github.com/garethgeorge/backrest

Re: Tarsnap is cozy

#70
post #36

Earlier quoted context omitted.

OP here, thanks for using the cost estimator! [1] I'm glad you got some use out of it. I use Backblaze B2 myself for most of my general purpose backup needs. It's actually $6/month, I believe. Tarsnap fills but one niche in my overall system. It's a very important niche for which I haven't found any other providers who do anything similar (keyfiles, prepaid, borderline anonymous etc), but it's not where I store the v…

I just don't really understand what the niche is. If you have a tiny bit of data that you want to keep backed up and rarely access, you can encrypt it with any number of easy command-line or GUI tools and upload it to Google Drive, Dropbox, or anywhere else with a free tier. If it's securely encrypted, there's no reason to care that the storage provider knows who you are. Tarsnap definitely has nerd appeal, but I can…

I actually do that. I have two folders one synced up to Dropbox and another to iCloud and they are encrypted with Cryptomator. But I back up the mounted folders i.e data to tarnsap. Besides those encrypted files (fragments?) of Cryptonator were messed up by both iCloud and Dropbox over the last few years. Gratned it could be Restic and Borg (I use these two as well for larger data sets), but for very small data tarsnap has woked well for me and prices are tiny. For a larger data set, I won't pick tarsnap.

If there's an simple but "solid" GUI backup tool with (true) PAYG I'd migrate away from Tarsnap, but there isn't one.

Post reply on HN