Update on 1/28 service outage

#1

Post by traviskuhl » Fri, Jan 29, 2016, 4:07 PM UTC

Update on 1/28 service outage
github.com

Re: Update on 1/28 service outage

#2

Post by bjacobel » Fri, Jan 29, 2016, 4:14 PM UTC

Not much detail here. A more thorough postmortem would give me more confidence they can recover from another similar issue. Hoping to see one soon.

Re: Update on 1/28 service outage

#3

Post by frik » Fri, Jan 29, 2016, 4:14 PM UTC

You can see the cascade effect on their status page graphs: https://status.github.com/

Re: Update on 1/28 service outage

#4

Post by [deleted] » Fri, Jan 29, 2016, 4:20 PM UTC

[deleted]

Re: Update on 1/28 service outage

#5

Post by Zikes » Fri, Jan 29, 2016, 4:21 PM UTC

bjacobel wrote: post #2

Not much detail here. A more thorough postmortem would give me more confidence they can recover from another similar issue. Hoping to see one soon.

I agree that a postmortem would be great, but it's good PR for companies to quickly put out statements like this to admit fault and maintain customer trust.

Re: Update on 1/28 service outage

#6

Post by tommoor » Fri, Jan 29, 2016, 4:22 PM UTC

This post makes it sound like Github has it's own data centers and power infrastructure structure, this is definitely news to me.. I'd presumed co-lo at best.

Re: Update on 1/28 service outage

#7

Post by Zikes » Fri, Jan 29, 2016, 4:23 PM UTC

[deleted] wrote: post #4

[deleted]

I'm starting to think that people should mirror their packages to BitBucket as a rule, and that package managers should round robin/flip a coin between the two, or use whichever is available in case of outages.

Re: Update on 1/28 service outage

#8

Post by smaili » Fri, Jan 29, 2016, 4:23 PM UTC

It's always scary when a cloud service you rely on goes down but great to see GitHub recover. Well done!

Re: Update on 1/28 service outage

#9

Post by anon987 » Fri, Jan 29, 2016, 4:25 PM UTC

bjacobel wrote: post #2

Not much detail here. A more thorough postmortem would give me more confidence they can recover from another similar issue. Hoping to see one soon.

Yep, I think most of these post-mortems from any company are pointless from a technical perspective. It's 4 paragraphs that boils down to "someone did something wrong and we'll make sure it doesn't happen" with zero specifics.

There's no point in reading these because there's no technical information. Stuff like this is something you sent to your customer because they want root cause.

Re: Update on 1/28 service outage

#10

Post by bhaak » Fri, Jan 29, 2016, 4:26 PM UTC

"Millions of people and businesses depend on GitHub"

Well, we shouldn't depend on it so much.

I shudder at the thought what an outage of GitHub would mean for our company. This time, we were lucky as it was during the night in Europe.

Unfortunately, I don't have the power to test this scenario in our company.