Earlier quoted context omitted.
I haven’t checked the pricing in a long time but you can use Tarsnap also if you have to backup only 7.3kb (okay I might ne exaggerating here but you get the drift) and pay for only that much. You can’t do that with Wasabi et al. Also it’s really simple and does what it says it does, nothing more, nothing less. In today’s everything convoluted and bloated world this is a luxury imho. The GUI app is also quite good an…
Backblaze has kept my ‘shutdown two years ago’ machine data without issue. What problems did you have with them (or did others have)?
Tarsnap outage postmortem
101–110 of 338 posts
Re: Tarsnap outage postmortem
#102Based on the description it sounds like it should be relatively easy to test this recovery process on a regular basis, to catch any lingering bugs and evaluate the recovery time. As they say, the only backups are the ones you have tested.
As someone who just discovered my DR process does not work by testing it, 100% this. The only plan that is likely to work is a repeatable tested one.
For example in both trains and cars, thanks to anti-lock braking, the correct way to stop the vehicle ASAP is to brake just like normal but as hard as you can, the computers will automatically solve the much trickier problem of turning your input into maximum deliverable braking force by periodically releasing brakes on sticking wheels.
If you run a fire drill, it's surprisingly difficult to get employees to use fire doors that they're used to finding alarmed and unusable. Even though intellectually they know that, say, the door at the bottom of the stairwell is a fire door, with crash bars and leads directly to the outside world, and this is a fire drill, they are likely to (for example) exit on a higher floor and go through a chokepoint lobby, as they would normally, instead of following this safer path that is emergency only. Sadly it is hard to fix buildings after construction if they were designed with such "unused" emergency exits.
For a backup process, having restoring machine images be a service that is sometimes, though not constantly, used anyway for some other reason, is a good way to be comfortable with how it works, that it works, etc. At work for example we routinely test upgrades on test servers restored from a recent backup. Restore serviceA to testA, apply upgrade, discover upgrade completely ruins the service, throw testA away and report this upgrade is garbage. But in the process we gained confidence in the restore process, infrastructure people instead of trying to recall something they only ever did in a drill, when things go badly wrong are very used to this procedure because they do it "all the time".
Re: Tarsnap outage postmortem
#103Earlier quoted context omitted.
The prices are absurdly high if your use-case is storage of large volumes of data that regularly change. It wouldn't be sensible to use Tarsnap for that, and you probably want to use one of the bulk backup services instead. Tarsnap makes a lot of sense when you benefit from the encryption and (especially) de-duplication features that it offers. For me, all of my most important personal and business data, from multipl…
> Tarsnap makes a lot of sense when you benefit from the encryption and (especially) de-duplication features that it offers. Can you provide more detail why you think so? I don't believe there is any use case in which tarsnap makes sense, other than maybe some Plan-C backup solution which you fall back on in the highly unlikely event that neither Plan-A nor Plan-B worked. Concretely, what benefits does tarsnap offer…
Re: Tarsnap outage postmortem
#104>The process of recovering the EC2 instance state consists of two steps: First, reading all of the metadata headers from S3; and second, "replaying" all of those operations locally. (These cannot be performed at the same time, since the use of log-structured storage means that log entries are "rewritten" to free up storage when data is deleted; log entries contain sequence numbers to allow them to be replayed in the…
How, exactly, is that a good thing?
Re: Tarsnap outage postmortem
#105Earlier quoted context omitted.
The HTML rendition in this case did it’s best to be hard to read. It’s not impossible, and likely just a fault of whatever list thing is used, but it could be better, and it’s nice if people let you know as such right?
How can simple black on white text with a lisible font be hard to read?
That’s barely possible to read on a high dpi screen, but fairly uncomfortable.
Re: Tarsnap outage postmortem
#106Earlier quoted context omitted.
Backblaze has kept my ‘shutdown two years ago’ machine data without issue. What problems did you have with them (or did others have)?
Backblaze has a policy of allowing backups of external disks, but the disks have to be connected at least once every 30 days, or they'll delete the backups. I understand they want to avoid abuse, but the lack of any grace period, or ability of support to ad an override, really soured the service for me.
Re: Tarsnap outage postmortem
#107Earlier quoted context omitted.
Backblaze has kept my ‘shutdown two years ago’ machine data without issue. What problems did you have with them (or did others have)?
Backblaze has a policy of allowing backups of external disks, but the disks have to be connected at least once every 30 days, or they'll delete the backups. I understand they want to avoid abuse, but the lack of any grace period, or ability of support to ad an override, really soured the service for me.
Re: Tarsnap outage postmortem
#108>The process of recovering the EC2 instance state consists of two steps: First, reading all of the metadata headers from S3; and second, "replaying" all of those operations locally. (These cannot be performed at the same time, since the use of log-structured storage means that log entries are "rewritten" to free up storage when data is deleted; log entries contain sequence numbers to allow them to be replayed in the…
Cost and reliability?
* Using S3 as a simple database is generally going to be much cheaper than RDS.
* If you turn on point in time restore, then losing data stored in S3 is not a possibility worth worrying about on a practical level for most people. RDS replication is easy enough to use, but adds more cost and a little bit of extra infra complexity.
Re: Tarsnap outage postmortem
#109Earlier quoted context omitted.
> Tarsnap makes a lot of sense when you benefit from the encryption and (especially) de-duplication features that it offers. Can you provide more detail why you think so? I don't believe there is any use case in which tarsnap makes sense, other than maybe some Plan-C backup solution which you fall back on in the highly unlikely event that neither Plan-A nor Plan-B worked. Concretely, what benefits does tarsnap offer…
My post was specifically addressing the comments around cost.
Re: Tarsnap outage postmortem
#110Earlier quoted context omitted.
I backup around 2TB with Restic, also tried locally with Borg. The size is nearly the same. Sadly, I can’t even test with Tarsnap! (absurd pricing for 2TB).
> absurd pricing for 2TB Well, it can't be that ba.. $0.25 x 2000 = $500 Yikes. And this is without BW costs. At $500/M you can just rent a dedicated physical server with a lot of HDDs and still have money left for your favourite pumpkin latte. For comparison rsync.net says it's $0.015 per GB/Mo, for 2TBs that's $30/m and no BW costs.
[0] https://www.scaleway.com/en/object-storage