Live data from Hacker News

Tarsnap outage postmortem

mail.tarsnap.com

321–330 of 338 posts

Re: Tarsnap outage postmortem

#321
post #154

Earlier quoted context omitted.

I'm renting $15/mo 2TB atom machine from OVH/kimsufi as second target for backups. Now that I think about it... some kind of micro-distributed backup server (throw on few of your machines, auto-replicate between) would be a neat project...

It's not even that neat. Just slap rsync/syncthing to the backup dir.

I do use syncthing on NAS + remote cheapo server for my day to day stuff, and bareos for rest.

It's just PITA to add another instance.

Re: Tarsnap outage postmortem

#322
post #277
post #9

Earlier quoted context omitted.

Why would I use your service over restic? God bless you Colin, but reading this, it appears you're the only one in charge of the infrastructure for this service. I'm glad you're clear about no SLA, but this seems like a big liability between me and my backups.

Uptime isn't an important property of a backup solution, so I'm not sure where the expectation comes from?

It sure should be up when you need it, exactly at the time you need it.

Re: Tarsnap outage postmortem

#323
post #281

Earlier quoted context omitted.

For the record, I'm happy to live in a world where rsync.net exists. I've pointed quite a few customers in your direction over the years, when tarsnap hasn't been suitable for their needs for a variety of reasons.

They make a good pairing. I backup my ZFS NAS to rsync.net for all my media and Tarsnap for all my documents/critical things.

I am only using rsync.net at the moment, more specifically with the discounted "borg" mode without an explicit full shell.

Your comment sounds like tarsnap is more secure (in terms of longevity) than rsync.net. Is this true? If yes, why?

Genuine question, because I'm using rsync.net for my critical stuff and would gladly move to tarsnap if appropriate.

Re: Tarsnap outage postmortem

#324
post #281

Earlier quoted context omitted.

They make a good pairing. I backup my ZFS NAS to rsync.net for all my media and Tarsnap for all my documents/critical things.

I am only using rsync.net at the moment, more specifically with the discounted "borg" mode without an explicit full shell. Your comment sounds like tarsnap is more secure (in terms of longevity) than rsync.net. Is this true? If yes, why? Genuine question, because I'm using rsync.net for my critical stuff and would gladly move to tarsnap if appropriate.

For me it's three fold.

First is that my usage of Tarsnap pre-dates my usage of rsync.net so it's been the primary backup of my home directory since ~2010. I haven't felt a need to change it and the 2 occasions I needed to restore it everything worked perfectly. i.e don't fix what isn't broken.

The second is that while I can in theory restore from rsync.net I actually never have... this is more a testament to the relability of ZFS though I guess and local snapshots have always been enough. That said the convenience of send/recv is sort of awesome.

Lastly is that I don't use ZFS on my client machines. If I did I would probably consider rysnc.net for everything.

So it's not really an explicit I think one is more secure or durable than the other it's that Tarsnap has fulfilled my DR needs successfully for a long time and I have come to trust it to do so in the future.

Re: Tarsnap outage postmortem

#325
post #245

Earlier quoted context omitted.

" Please don't complain about tangential annoyances—e.g. article or website formats, name collisions, or back-button breakage. They're too common to be interesting. " https://news.ycombinator.com/newsguidelines.html

Shame. It actually spawned an interesting discussion about reader mode, ASCII emails, browser rendering engines and the meeting of old and new. And I learnt things.

I believe you and yes those are upsides - but we have to moderate for how these things work in general, and in general the downside of such digressions is much greater.

Re: Tarsnap outage postmortem

#326
post #252

Earlier quoted context omitted.

> God bless you Colin, but reading this, it appears you're the only one in charge of the infrastructure for this service Hence the toddler.

I am really confused by the communication thread and am interpreting that the toddler is somehow in charge of running the infrastructure as a joke. Yet I can’t see it as either a joke or serious. I’m a native English speaker but sometimes I swear I’m losing grasp on communication in the Internet age and am sincerely trying to understand this all.

My toddler runs https://rangerovers.pub and it mostly holds up okay. He's not great at yaml because he can't really read so the significant whitespace is a problem, but he knows how to run the backups and ensure the mail handler isn't choking on all the Russian spammers. We try to limit his screen time though so he's only on for the 15-minute maintenance window. The Aprilscherz frontend for Docker is a big help.

Re: Tarsnap outage postmortem

#327
post #295

Earlier quoted context omitted.

Don't let the CTO be a scapegoat. Entire executive leadership, all board members and the 5 largest shareholders.

We have just written the Sarbanes-Oxley for the tech regulation industry- all we need now is a congresswoman and a senator and a good acronym Secure Technology Oversight for Corporate Software STOCS Act here we come ! Edit : yeah I could not get the K in ... that's hard

Korporate.

Re: Tarsnap outage postmortem

#328
post #257

Earlier quoted context omitted.

Please see https://news.ycombinator.com/item?id=36897868 and please stop posting in the flamewar style to HN. Regardless of how right you are / how much smarter you are or feel you are, it's exactly what the rules here ask you not to do. We're trying for a very different quality of conversation here. https://news.ycombinator.com/newsguidelines.html

You're right, that was uncalled for. I apologize for lowering the quality of site.

Thanks for the kind reply.

Re: Tarsnap outage postmortem

#329

Earlier quoted context omitted.

It's a pretty well-known fact for years that tarsnap is basically a one-man show, and yet Colin has managed to provide fantastic service so far. Sometimes having ppl who built the service also managing it is actually a big plus, compared to other services where you first have to fight through outsourced & underpaid support that's limited to template answers, only to finally get some "engineer" who got that job 2 mont…

And to be frank, I've seen plenty of mission-critical services at $bigco which may have had a team of engineers working on them, but the core functionality was maintained, understood, and supported by effectively one senior engineer. If anything went wrong, the supporting junior staff might have been able to fix reasonably simple stuff, but there was essentially one person who understood the system deeply enough to h…

Absolutely.

Early in my career, I became the second person able to support and operate a system that was public facing and responsible for billions of dollars of activity that mattered to many individuals and stakeholders. The entire team retired over a period of six months, after giving the folks in charge a year or more notice. After about 12 weeks, I was the sole guy, training a 4-5 new people.

We’re all probably using a service like this. As demonstrated by Twitter, well engineered systems can persist, even without proper care and feeding, until they don’t.

Re: Tarsnap outage postmortem

#330
post #274

Earlier quoted context omitted.

Did I fall into a timewarp into the 70's? How on Earth, by what sane standard, is a Postgres instance too complex? If you're `fopen`ing files as a "database" you are wasting your time and lowering the world's economic productivity. Complex is Greenspunning a database and having it blow up in your face and cause a twenty-six hour outage. You never hear about such things with Postgres because Postgres is rock-solid.

[flagged]

Could you please stop posting unsubstantive comments and flamebait? You've unfortunately been doing it repeatedly. It's not what this site is for, and destroys what it is for.

If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful.

Post reply on HN