Live data from Hacker News

GitHub is degraded/down

githubstatus.com

161–170 of 171 posts

Re: GitHub is degraded/down

#161

Earlier quoted context omitted.

As a side note, gotta say, scare-quoting "collaborate" seems like a pretty weird flex in a time when everybody's stuck working from home whether they prefer it that way or not. Maybe you prefer to work entirely within a silo, rarely dealing with colleagues in the course of your day-to-day. Most don't, nor should they. Most kinds of work, even the kinds of work that we do, really aren't better done that way. I worked…

In my experience, there are a lot of developers who get really uncomfortable once they start having more developers working on their code. There’s a lot of effort which has to be done documenting the code: Either by explaining how the code runs to other engineers on the team, or by having documentation showing how the code is arranged and runs, something a lot of engineers are not very good at doing. It takes a lot o…

I mean, that's fair, but growing past that is also part of learning to be an effective engineer. It's not always a comfortable process - it sure wasn't at first for me! - but that's often true of any kind of growth.

I think it's better described as a skill than as a talent, too. The former is learned, the latter innate, and as someone with absolutely none of the latter, I can attest that the former is attainable nonetheless.

Re: GitHub is degraded/down

#162

Earlier quoted context omitted.

> Who's to say? And then you go on to say it. I'm glad that SourceHut exists, and I like many of its principles, and it's probably better designed too, but walking into a thread where someone is having an outage and then claiming that you'd do much better is in poor taste no matter how good you are or how many of your services work offline.

I responded directly to someone who said they were considering alternatives, and wouldn't've otherwise.

Right, and I think it is great to bring up how your service can handle outages better than GitHub would due to it being decentralized. The part I have issue with is saying that you'd do better than GitHub about keeping your site up, pointing to the issue that they are in the middle of resolving–that just seems like kicking them while they're down, especially since you haven't actually shown that you can do better. (Yes, you have good uptime in the past, but I don't see what's stopping the power going out to some of your servers, or you pushing a bug into production, or any number of other things that shouldn't go wrong but often do, especially as the number of users increases.)

Re: GitHub is degraded/down

#163

Earlier quoted context omitted.

So you're comparing your scalability with a company with over 40m users and 100m repos. Can you talk about the geographic distribution of your 10 servers?

I would like to remind you of my earlier point: SourceHut is not the same scale as GitHub. This does not change the fact that SourceHut is faster and more reliable. We have an advantage - fewer users and repos - but still, that doesn't change the fact that we're faster and more reliable . This has been objectively demonstrated as a numerical fact: https://forgeperf.org And yes, 9 of those servers are in Philadelphia…

> This has been objectively demonstrated as a numerical fact:

While your tests are indeed objective, I don't think they're very useful. For example, why does your performance test ignore caching?

GitHub's summary page loads 27KiB of data for me unauthenticated, which is about 6% of the 452KiB you're displaying in your first table. The vast majority of developers who browse GitHub will not be loading 452KiB of static assets every single page load.

Anecdotally, GitHub's "pjax" navigation feels about as fast as SourceHut on my aging hardware.

Re: GitHub is degraded/down

#164

Earlier quoted context omitted.

I responded directly to someone who said they were considering alternatives, and wouldn't've otherwise.

Right, and I think it is great to bring up how your service can handle outages better than GitHub would due to it being decentralized. The part I have issue with is saying that you'd do better than GitHub about keeping your site up, pointing to the issue that they are in the middle of resolving–that just seems like kicking them while they're down, especially since you haven't actually shown that you can do better. (Y…

>what's stopping the power going out to some of your servers

Redundant power supplies

>pushing a bug into production

Nothing, but again, SourceHut is demonstrably better in this regard: because it's distributed, a bug in production would only affect a small subset of our system, and the system knows how to repair itself once the bug is fixed.

And I don't think I need to apologise for kicking Golaith while he's down. Someone said they want alternatives, so I pitched mine with specific details of how it's better in this situation, and that doesn't seem wrong to me. I would invite my competitors to do the same to me. We should be fostering a culture of reliability and good engineering - and if I didn't hold my competitors accountable, who will? "Here's an alternative" has more teeth than "I wish this was better."

Re: GitHub is degraded/down

#165

Is it just me, or has Github's quality of service been continually degrading over the past several months? What is going on internally? Is this because of the Microsoft acquisition? Increased usage? An internal transition to Azure? ...is it time to move away from Github?

Positive side for such downtime, turns out people gather to look at your landing/home page now just to see if the service is up. Can the cost of a downtime be effective to grab few customers for your new feature just published on your homepage?

Re: GitHub is degraded/down

#166
post #150

Earlier quoted context omitted.

You can test Cloudflare at https://cloudflare-test.judge.sh/#snazz.xyz

So ORD is probably just higher-capacity, which is why free plan users like me are getting routed to it?

Most likely, the other/normal DC is likely either overloaded or has part of its hardware under maintenance.

Re: GitHub is degraded/down

#167

Earlier quoted context omitted.

I would like to remind you of my earlier point: SourceHut is not the same scale as GitHub. This does not change the fact that SourceHut is faster and more reliable. We have an advantage - fewer users and repos - but still, that doesn't change the fact that we're faster and more reliable . This has been objectively demonstrated as a numerical fact: https://forgeperf.org And yes, 9 of those servers are in Philadelphia…

> This has been objectively demonstrated as a numerical fact: While your tests are indeed objective, I don't think they're very useful. For example, why does your performance test ignore caching? GitHub's summary page loads 27KiB of data for me unauthenticated, which is about 6% of the 452KiB you're displaying in your first table. The vast majority of developers who browse GitHub will not be loading 452KiB of static…

Even with caching, SourceHut is a lot smaller than that. SourceHut benefits from caching, too - the repo summary page comes from 2 requests and 29.5K to 1 request and 5.7K with a warm cache. And in many cases, the cache isn't the bottleneck, either - dig into the Lighthouse results for specific pages to see a more detailed breakdown.

Re: GitHub is degraded/down

#168
post #6

Earlier quoted context omitted.

They've been pushing lots of features in the platform so not surprising that it's a bit unstable now.

In an ideal world, pushing new features would have no impact on stable mature features like browsing files, comment threads, etc

Too bad this is the real world.

Re: GitHub is degraded/down

#169

At my organization, we saw an uptick in timeouts spanning vendors that started at the same time - 6:55am PST. Makes me think there's an internet wide event occurring right now

We were experiencing issues with Netlify at 6:15am PST.

Re: GitHub is degraded/down

#170

As a huge fan of JetBrains products (I use WebStorm, PyCharm, DataGrip in roughly equal parts) I am considering: https://www.jetbrains.com/space/ Anybody else got experience with that and/or TeamCity?

We use TeamCity and can't say I have any complaints. We utilize the tagging feature pretty heavily in our deployments and source all artifacts from TeamCity builds. The builds also post success or failure to Gerrit.
Post reply on HN