Tarsnap outage postmortem

Post by saagarjha » Fri, Jul 28, 2023, 7:07 AM UTC

idlewords wrote: post #215

cowsandmilk wrote: post #158

Earlier quoted context omitted.

It isn’t necessarily about Colin quitting. Key person gets hit by bus is also always a concern. You can say someone will pick it up, but I know nothing of whether such plans are in place. Does the person who would inherit the business have the know how to sell it? Is there enough documentation in place for a transfer of assets to be successful?

This is how that scenario shakes out: 1. Key person gets hit by bus 2. You see the black bar on Hacker News and learn the sad news 3. You go download all your data from the service, which is still up because there is no bus access to data centers. 4. You feel like a jerk for all your creepy "hit by bus" talk. 5. A few weeks later, some VC-funded operation with multiple employees you depended on disappears overnight w…

Post by AtNightWeCode » Fri, Jul 28, 2023, 8:01 AM UTC

mrweasel wrote: post #61

AceJohnny2 wrote: post #39

Earlier quoted context omitted.

It's not about suing, but defining expectations about how you can rely on a service. For example, my team has people across the world for HW bringup, so we can't allow our code hosting or CI to be down for more than a few hours. Of course, backups have different uptime requirements, but as for everything, it's a tradeoff between features, of which an SLA is one. Tarsnap's features are granularity of cost, reliability…

> It's not about suing, but defining expectations about how you can rely on a service. Meeeeh, my ISP cut of around 100+ fiber connections in my town and spend three weeks fixing it. My neighbor have business line, there's an SLA on those that among other things, require them if reestablish his connection within 3 - 5 hours. It took them over 500 hours, so that SLA is useless for anything but forcing compensations. T…

Post by switch007 » Fri, Jul 28, 2023, 8:26 AM UTC

TheDong wrote: post #71

switch007 wrote: post #53

Earlier quoted context omitted.

A is the correct kind of HTML use for the body of plain text email ? It looks like paragraphs of text to me. is far more appropriate That isn’t apple’s problem, nor mine.

pre is the only correct element to use since in many emails, the exact formatting and linebreaks and such are important. For example, a code review on a mailing list can only make sense with the linebreaks and spacing preserved. However, as you knew to try, there is "reader mode", which is meant to heuristically ignore the exact html in order to display textual content. Firefox's reader mode has no trouble figuring o…

Post by LinAGKar » Fri, Jul 28, 2023, 10:53 AM UTC

cperciva wrote: post #3

blinks Ok, I really wasn't expecting this to land at the top of HN. I'd love to stick around to answer any questions people have, but it's 10PM and my toddler decided to go to bed at 5PM... so if I'm lucky I can get about 4 hours of sleep before she decides that it's time to get up. I'll check in and answer questions in the morning.

Post by Aachen » Fri, Jul 28, 2023, 11:59 AM UTC

rssoconnor wrote: post #140

patrec wrote: post #87

Earlier quoted context omitted.

> Tarsnap makes a lot of sense when you benefit from the encryption and (especially) de-duplication features that it offers. Can you provide more detail why you think so? I don't believe there is any use case in which tarsnap makes sense, other than maybe some Plan-C backup solution which you fall back on in the highly unlikely event that neither Plan-A nor Plan-B worked. Concretely, what benefits does tarsnap offer…

I use tarsnap because the asymmetric crypto means I can give my cron job authorization to create backups, but it doesn't have authorization to read or delete(!) backups. This ability is critical to prevent a compromised system from having its data wiped and having all backups wiped as well. I haven't been able to figure out how to do this in any other system. But if someone has a tutorial, I am all ears.