Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GitHub is degraded/down

githubstatus.com

Post reply on HN
Print view View original

151–160 of 171 posts

Re: GitHub is degraded/down

#151

Post by bdcravens »

ddevault wrote: post #149
bdcravens wrote: post #145

Earlier quoted context omitted.

So you're comparing your scalability with a company with over 40m users and 100m repos. Can you talk about the geographic distribution of your 10 servers?

I would like to remind you of my earlier point: SourceHut is not the same scale as GitHub. This does not change the fact that SourceHut is faster and more reliable. We have an advantage - fewer users and repos - but still, that doesn't change the fact that we're faster and more reliable . This has been objectively demonstrated as a numerical fact: https://forgeperf.org And yes, 9 of those servers are in Philadelphia…

I was considering your claim:

> we use a distributed architecture

> SourceHut is faster

I wasn't questioning that some of the web features are fast. I'm sure when Github was 10 servers their pages were fast too. I suspect if I threw Gitlab on a 9-server cluster on AWS they'd also be quick.

Re: GitHub is degraded/down

#152

Post by sneak »

hellepardo wrote: post #2

Is it just me, or has Github's quality of service been continually degrading over the past several months? What is going on internally? Is this because of the Microsoft acquisition? Increased usage? An internal transition to Azure? ...is it time to move away from Github?

I set up a self-hosted Gitea this year and moved my repos over and couldn't be happier with it. It's faster than GitHub, clones the GitHub design/UI so that everything's where I expect it to be, has a dark mode, and supports U2F. It's easy to deploy, back up, and maintain, the Gitea devs have done a great job.

It's much less complicated (both from an admin standpoint, as well as a UI standpoint) than GitLab. I paired it with a Drone installation (also self-hosted) for CI and (sometimes) CD.

It all works great, and is way easier than I thought. If there's downtime, I'm (usually) in control of when or how long, as I have root on the box.

I'm also not giving my money to a giant military contractor (Microsoft, the owners of GitHub) any longer, which is a huge deal for me from a personal moral standpoint (YMMV).

Re: GitHub is degraded/down

#153

Post by ddevault »

bdcravens wrote: post #151
ddevault wrote: post #149

Earlier quoted context omitted.

I would like to remind you of my earlier point: SourceHut is not the same scale as GitHub. This does not change the fact that SourceHut is faster and more reliable. We have an advantage - fewer users and repos - but still, that doesn't change the fact that we're faster and more reliable . This has been objectively demonstrated as a numerical fact: https://forgeperf.org And yes, 9 of those servers are in Philadelphia…

I was considering your claim: > we use a distributed architecture > SourceHut is faster I wasn't questioning that some of the web features are fast. I'm sure when Github was 10 servers their pages were fast too. I suspect if I threw Gitlab on a 9-server cluster on AWS they'd also be quick.

Not geographically distributed, but distributed in the sense that different responsibilities of the overall application are distributed among different servers, which can fail independently without affecting the rest. Additionally, the mail system on which many parts of SourceHut relies is distributed in the geographical sense, among the hundreds of thousands of mail servers around the world which have standard and 50-year-battle-tested queueing and redelivery mechanisms built in.

And yes, throwing GitLab on a 9 server cluster on AWS might be fast. But, I'm ready to bet you that SourceHut will be faster than it still, and I have a ready-to-roll performance test suite to prove it. And I know that SourceHut is faster than GitLab.com and GitHub.com, and every other major host, and you don't have to go through the trouble of provisioning your own servers to take advantage of SourceHut's superior performance.

Re: GitHub is degraded/down

#154

Post by tracker1 »

Wowfunhappy wrote: post #143
dx034 wrote: post #16

Earlier quoted context omitted.

Maybe it's the demand side? With remote work, the intensity of usage went up at least in our company as we rely more on written communication. At the same time, some people will use the time to start side projects or get into programming.

On the other hand, shouldn't there be a productivity drop with so many people working from home while their kids also aren't in school? I'd expect that to offset any increase in demand—after all, Git isn't Slack; remote work shouldn't cause people to push all that much more often, right?

Depends... I'm getting about 40% more done, with fewer interruptions, and not having a couple hours of commute and lunch driving.

Re: GitHub is degraded/down

#155

Post by tracker1 »

hansdieter1337 wrote: post #124
hellepardo wrote: post #2

Is it just me, or has Github's quality of service been continually degrading over the past several months? What is going on internally? Is this because of the Microsoft acquisition? Increased usage? An internal transition to Azure? ...is it time to move away from Github?

1) Microsoft took over 2) M$ migrates some ADO (Azure DevOps) features to Github (e.g., Github Actions) 3) If Github was not on Azure before M$ bought it (very likely, but needs citation) they will probably migrate to Azure at some point

I'm pretty sure Github Actions work predates the MS acquisition... I'm also pretty sure that they are trying to align the backend systems more to Azure, but have no insight into how much of that took place.

The fact that you used "M$" indicates that you are predisposed to blame Microsoft for actions that are likely not from the parent, and discount any changes from the top down that have occurred within MS. And while I have a lot of issues with MS and Windows in particular, MS today is not the same as MS even a decade ago.

Re: GitHub is degraded/down

#156

Post by DenseComet »

sailingparrot wrote: post #103
wolco wrote: post #98

Earlier quoted context omitted.

That would point to Azure hosting. Anyone notice a similiar pattern?

Github is still hosted on AWS though afaik.

As of the end of 2017, they were using their own datacenters.

https://github.blog/2017-10-12-evolution-of-our-data-centers...

Re: GitHub is degraded/down

#158

Post by strenholme »

derefr wrote: post #55
EvanAnderson wrote: post #43

Earlier quoted context omitted.

The "we deploy production directly from Github" pathology is the one I'm talking about. Not being able to "collaborate" for a brief outage is fine. Not being able to deploy code isn't.

You gotta CI/CD from somewhere . Whatever that somewhere is, it can go down. You can, of course, override your CI/CD and do a manual deploy. It's not a matter of "can." It's a matter of not fully understanding all the checks the CI/CD system does to the code, and all the build steps for prod builds, and therefore not having the confidence to deploy to prod without CI/CD holding your hand. (Which I don't at-all blame…

Well, the place I run my CI tests is on a VMware virtual machine, e.g.

  #!/bin/bash -e
  git clone https://git.example.com/myrepo
  cd myrepo
  sh do.tests
If the tests are automated enough to run with every Git checkout, they can be easily enough be run by hand too.

The real world script I use is a little more complicated, because the code base in question is two decades old so I need to make some changes to how the code looks to the tests so that the tests can run.

I once worked for a company which had a series of tests which took eight to ten hours to run. Running those tests with every single Git checkin was out of the question; we instead used cron to run the tests every night.

Re: GitHub is degraded/down

#159

Post by strenholme »

throwanem wrote: post #95
EvanAnderson wrote: post #43

Earlier quoted context omitted.

The "we deploy production directly from Github" pathology is the one I'm talking about. Not being able to "collaborate" for a brief outage is fine. Not being able to deploy code isn't.

As a side note, gotta say, scare-quoting "collaborate" seems like a pretty weird flex in a time when everybody's stuck working from home whether they prefer it that way or not. Maybe you prefer to work entirely within a silo, rarely dealing with colleagues in the course of your day-to-day. Most don't, nor should they. Most kinds of work, even the kinds of work that we do, really aren't better done that way. I worked…

In my experience, there are a lot of developers who get really uncomfortable once they start having more developers working on their code. There’s a lot of effort which has to be done documenting the code: Either by explaining how the code runs to other engineers on the team, or by having documentation showing how the code is arranged and runs, something a lot of engineers are not very good at doing.

It takes a lot of talent to do collaboration well, because it means having to let go of the code being “my code”, and it means having a lot of people skills, such as the ability to empathize with how someone not familiar with the code will see things, and how to write documentation about the code structure so that things can be in maintainable “boxes”.

Re: GitHub is degraded/down

#160

Post by samanthalee233 »

nimbius wrote: post #114
hellepardo wrote: post #2

Is it just me, or has Github's quality of service been continually degrading over the past several months? What is going on internally? Is this because of the Microsoft acquisition? Increased usage? An internal transition to Azure? ...is it time to move away from Github?

you could move to gitlab, but from what im hearing the pricing is higher than github (is this still true?) Barring that you always have the tried and true (and for some reason abhorred by start-ups) option of running your own gitea or gitlab instance. Its not hard, and most of this stuff can be done in dockerless containers if you want. If cloud servers are getting "overloaded" as some commenters say, you could even…

GitLab community advocate here, just wanted to share the most up to date GitLab pricing information: https://about.gitlab.com/pricing/ Thanks!
Post reply on HN