Live data from Hacker News

GitHub Down with a 503

status.github.com

51–60 of 89 posts

Re: GitHub Down with a 503

#51
post #6

Random somewhat on-topic question. How does one measure ones technical capability to keep up services that are more available and dependent than third party solutions? I mean obviously starting out, almost everybody can benefit from doing a lot of stuff not in-house, but at some point you'd be able to do better yourselves. But then maybe for certain services from certain providers, raw technical capability never catc…

By the time you're talking about 99.9% uptime there is only 9 hours downtime a year so there isn't much wiggle room for failure. There is never a reason any one person should assume they can do better than a third party because you're likely sleeping 2920 hours a year. Basically, you're never good enough on your own. If you care about uptime you're going to need to pay good money for it.

Re: GitHub Down with a 503

#53
post #50

Somehow I opened the GitHub Status Page subconsciously early this morning, then everything suddenly became 503 in less than 10 minutes! I still have the tab opened now, saying "All systems operational"...

Seems like it's back now. (3:13 UTC)

Re: GitHub Down with a 503

#55
post #35

Exception percentage just got over 100%, new record! (?) http://i57.tinypic.com/4hphtd.png

It is now up to 175% a couple minutes later (and 205% a couple minutes after that). The issue seems to be that it is actually currently more like 10000%, and is being averaged down throughout the month to a mere 100-200%. What is this even measuring?!

If I were implementing something tracking the rate of exceptions a reasonable thing to measure it against is the total number of requests. In such a system, if each request generated multiple exceptions you would have >100% exception rate.

Re: GitHub Down with a 503

#56
post #37

Could Github store mirrors of all repositories on some third-party service, like S3? Even if they were only updated every 24 hours and had a limited history for each repo, it still seems like it would be a really useful fallback that they could put up when things like this happen.

It's actually not that hard to set up your git to push to both Github and another repository at the same time.

Re: GitHub Down with a 503

#57
post #25

Earlier quoted context omitted.

When Github appears down, I come to HN to find out if it's affecting everyone :p

I first check their Twitter account, and then when there's nothing there about an outage (currently isn't), I come here.

https://status.github.com/ is perhaps more useful?

Re: GitHub Down with a 503

#60
post #36

Earlier quoted context omitted.

If you're using a cloud running test service like Travis CI pushing to GitHub can trigger your whole deployment process starting with running the unit and integration tests.

Now you deploying production hinges on a third party. What if you push a broken build that your tests didn't cover, and GitHub goes down when you have the patch prepared? What if Travis goes down? Third-party services being in the critical path for applying code to systems is a recipe for outages and other trauma. If it impacts you enough to come to HN and leave the comment I replied to, you're doing it wrong -- ther…

Two third parties, even.

Github's uptime is good enough that I don't see anything inherently awful about a workflow that makes development difficult when Github is down, but any production deployment process that strictly requires anything beyond a working internet connection and a working server to deploy to is insane.

Post reply on HN