I have zero evidence but I would not be surprised if they are busy absorbing Azure Devops into GitHub and that's causing some hiccups.
Has GitHub been down more since its acquisition by Microsoft?
81–90 of 149 posts
Re: Has GitHub been down more since its acquisition by Microsoft?
#82Earlier quoted context omitted.
I'm sure these down times are more about infrastructure problems than just database updates and I'm also sure that this infrastructure is a little more complexe than your site.
I'm sure their deployment process is way more complex than mine. I'm also sure that unlike me, they have engineers dedicated to managing the complexity. GitHub is a company that has blogged about how easy they have made deployment and how they deploy many times a day. I don't believe there are any compelling reasons why that ease of deployment shouldn't also extend to re-deploying previous versions of their service,…
It’s often not as easy as deploy -> immediate problem -> rollback. Problems can take a while to diagnose, or may cause some kind of poisoning that needs to be fixed (eg rebuild a lost or corrupt cache), or be in some part of the system that nobody knew was related (eg maybe someone deployed code that talks to a hitherto-unqueried accounting system and that worked fine at 4pm on Thursday but come 9am Monday it melts).
My point is that in big complex systems sometimes there is not a straight line between cause end effect. Sometimes there’s just effect and you need to work out the cause.
Re: Has GitHub been down more since its acquisition by Microsoft?
#83Anecdotally, yes. I almost never saw internal error pages before, now I've been seeing them every week for a few weeks. Some people need to be fired.
Re: Has GitHub been down more since its acquisition by Microsoft?
#84A number of the staff who left GitHub in protest of the ICE contract were senior SRE. I wonder if the data says anything about the dates of their departures...
Re: Has GitHub been down more since its acquisition by Microsoft?
#85What I don't get is how they can be down for hours on end. If I make a deploy that turns out to be buggy, I'll roll my site back to a stable previous version within minutes. Sure, that's not always possible if I've made incompatible database schema changes, but in my experience those are very, very rare (i.e. I almost always add db columns, and only rarely delete or rename columns - and when I do, I do so after those…
Which commit would you roll back to if the integer id of your comments table overflows? There are many bugs that are more complex than a buggy commit.
Re: Has GitHub been down more since its acquisition by Microsoft?
#86Re: Has GitHub been down more since its acquisition by Microsoft?
#87Ironically, Firefox tells me to stay away from your website. Which, I will. update: the website is back and well now from my end.
I can't reproduce, would you mind emailing me a screenshot of this? (I'm the author) colin at any of the domains in my bio.
I would have happily complied. On a welcome note, however, the backend changes made after the initial hickups brought the website back on its feet.
Re: Has GitHub been down more since its acquisition by Microsoft?
#88Taking the numbers at their face value this is a good exercise in some text book statistical test of hypothesis. Incidents: Before 89. After 126. What is the chance of this happening if the 'rate' of occurrence has not changed ? Assuming an unknown but constant Poisson rate, we get the probability of observing what has been observed to be 0.00225. A fortuitous thing about this test is that one does not need to know w…
Maybe Github has grown significantly since then. More complexity, more down time.
Re: Has GitHub been down more since its acquisition by Microsoft?
#89This is a thinly veiled advert, and tells you literally nothing concrete, apart from Github's status page is updated more. The article even admits this!
Re: Has GitHub been down more since its acquisition by Microsoft?
#90Looking at the chart, there were more warns but fewer downtimes so I'd say no. There was one giant downtime that happened, but I don't know why it happened so it's hard to say if there was a cause relevant to the acquisition. The other downtimes that happened seemed to be slightly longer in duration than before.