Earlier quoted context omitted.
"Thanks for being a beacon of light in the dark." That's well put. It makes me very happy to live in a world where tarsnap exists and is priced in picodollars.
For the record, I'm happy to live in a world where rsync.net exists. I've pointed quite a few customers in your direction over the years, when tarsnap hasn't been suitable for their needs for a variety of reasons.
Tarsnap outage postmortem
281–290 of 338 posts
Re: Tarsnap outage postmortem
#282>> So far such an outage has never occurred; but over time Tarsnap will become more tolerant of failures in order to minimize the probability that such an outage occurs in the future.
Re: Tarsnap outage postmortem
#283Tarsnap is undoubtedly expensive, but it also donates to various efforts! Neglecting the pricing, does Tarsnap have any advantage over Restic? Restic also deduplicates, using little data.
I mean.. you could purchase a cheaper service and also donate to various efforts. Bonus: Then you'd also be able to pick those efforts.
Re: Tarsnap outage postmortem
#284Earlier quoted context omitted.
Completely ignoring costs, can you name a single use case for which tarsnap would be better than Borg or restic on rsync.net?
Backing up anything Windows with more granularity than top level directories? Ugh. Try picking a choosing specific file types or file extensions from filesystems holding thousands of files. I ended up having to cobble together some god-awful pre-process powershell with multiple pipes just because restic fails to be able to grep using Windows reliably. :(
That is news to me. I backup almost a million files spread across 4 Windows devices, with heavy use of --files-from and --iexclude and it seems to work. What am I missing?
I agree that restic filtering options are pretty limited. Too limited, really. But what's there seems to work?
Re: Tarsnap outage postmortem
#285Earlier quoted context omitted.
[flagged]
Please see https://news.ycombinator.com/item?id=36897868 and please stop posting in the flamewar style to HN. Regardless of how right you are / how much smarter you are or feel you are, it's exactly what the rules here ask you not to do. We're trying for a very different quality of conversation here. https://news.ycombinator.com/newsguidelines.html
Re: Tarsnap outage postmortem
#286Earlier quoted context omitted.
It's not Colin's fault that you're using a browser that can't render an html rendition of an email which has been widely in use since before iOS existed. This is entirely Safari's fault for not having good compatibility with a common existing webpage format. Anyway, if you're the intended audience (someone using tarsnap), you also received a copy to your email address, where you can read the text with your email read…
A is the correct kind of HTML use for the body of plain text email ? It looks like paragraphs of text to me. is far more appropriate That isn’t apple’s problem, nor mine.
Re: Tarsnap outage postmortem
#287Earlier quoted context omitted.
> Tarsnap makes a lot of sense when you benefit from the encryption and (especially) de-duplication features that it offers. Can you provide more detail why you think so? I don't believe there is any use case in which tarsnap makes sense, other than maybe some Plan-C backup solution which you fall back on in the highly unlikely event that neither Plan-A nor Plan-B worked. Concretely, what benefits does tarsnap offer…
I use tarsnap because the asymmetric crypto means I can give my cron job authorization to create backups, but it doesn't have authorization to read or delete(!) backups. This ability is critical to prevent a compromised system from having its data wiped and having all backups wiped as well. I haven't been able to figure out how to do this in any other system. But if someone has a tutorial, I am all ears.
I've been musing on this subject all afternoon. I'm a user of Tarsnap, and I do find it expensive, in the sense that I would prefer to backup larger amounts of data for less amount of money. At the moment I backup photos separately from Tarsnap and in an adhoc way.
But I still cannot figure out a way to get all the benefits I get from Tarsnap from any other software solution.
* Must be usable under Nixos.
* Backups must be asymmetrically encrypted so that backups can be automated, yet a compromise of the system cannot immediately gain read authorization to arcived data.
* Backups must be append-only without further credentials, or otherwise prevent a compromised system from being able to delete existing archives.
* Deduplication between archives while still allowing archives to independently be deleted.
Using the ZFS snapshot functionality with rsync.net, for example, with Duplicity comes close. However, as I recall, duplicity wants to regular (typically monthly) full backups and then incremental backups from there. You cannot remove these full backups without deleting the entire month's worth of backups, and because the full backups are independently encrypted, there is (of course) no deduplication between full snapshots, even though the data is still likely largely the same. And because the snapshots are encrypted, it is impossible for the rsync.net storage to see or even know that large parts of the encrypted data is identical.
AFAICT there is really nothing else that does what Tarsnap does.
Re: Tarsnap outage postmortem
#288Earlier quoted context omitted.
How's the saying in every HN thread go? "Don't set your prices based on your costs, set your prices based on the value you deliver." or something like that. Tarsnap is a wonderful piece of software. You're paying for that. That said, is the value of "Tarsnap" worth the price difference from "Borg+rsync.net"? (Or Restic, I've been meaning to look into Restic). I'm not so sure. These days I'm a customer of rsync.net, n…
Completely ignoring costs, can you name a single use case for which tarsnap would be better than Borg or restic on rsync.net?
With regards to duplicity, Tarsnap does full deduplication across all backups for any given "machine", while still letting you independently remove any snapshots you like. i.e. no special "full snapshot" that must always be kept around, and no need for multiple full snapshots that have no deduplication between them.
Re: Tarsnap outage postmortem
#289Earlier quoted context omitted.
How's the saying in every HN thread go? "Don't set your prices based on your costs, set your prices based on the value you deliver." or something like that. Tarsnap is a wonderful piece of software. You're paying for that. That said, is the value of "Tarsnap" worth the price difference from "Borg+rsync.net"? (Or Restic, I've been meaning to look into Restic). I'm not so sure. These days I'm a customer of rsync.net, n…
rsync.net is also overpriced for strictly backup purposes. Make sure you do check out restic; it can use S3 or Backblaze B2 (I actually use both) as backends instead of something expensive like rsync.net. The value of these boutique storage services evaporates when you start using restic.
Run rsync to the target and forget is quite easy, though I admit rsync.net's deal is getting worse these days posing minimum usages here and there.
Re: Tarsnap outage postmortem
#290Earlier quoted context omitted.
> Tarsnap makes a lot of sense when you benefit from the encryption and (especially) de-duplication features that it offers. Can you provide more detail why you think so? I don't believe there is any use case in which tarsnap makes sense, other than maybe some Plan-C backup solution which you fall back on in the highly unlikely event that neither Plan-A nor Plan-B worked. Concretely, what benefits does tarsnap offer…
I use tarsnap because the asymmetric crypto means I can give my cron job authorization to create backups, but it doesn't have authorization to read or delete(!) backups. This ability is critical to prevent a compromised system from having its data wiped and having all backups wiped as well. I haven't been able to figure out how to do this in any other system. But if someone has a tutorial, I am all ears.
* Create a S3 bucket and enable versioning * Create a new user and give it only s3:PutObject on your new bucket * Create an auth keypair for that user and put it on your server
Now any server compromise that gets those keys can only add new data to your backup bucket, and can't read, overwrite, or delete any previous backup.
There's no dedup, so that could be a deal-breaker.
There's also no real encryption (though that shouldn't be too hard to add I guess). I don't really see the gain though. Anyone who compromises the server keys is blocked from reading by AWS permissions. Granted, that's not quite as reliable as good crypto for blocking reading, but on the deleting side, there's never going to be anything but the auth system of whatever solution you're using to block that.
I get that there's some applications out there where preventing data exfiltration is important enough to need strong crypto (though is that really important when we're talking about full compromise of your server, which gets the attacker direct access to the data anyways?), but I decided that the risk of failing to implement properly or full data loss due to losing the keys or them being corrupted wasn't worth the risk of blocking somebody who somehow compromised the AWS account security from being able to read backup data.