Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GitHub was down

status.github.com

Post reply on HN
Print view View original

111–120 of 129 posts

Re: GitHub was down

#111

Post by humaid »

mindfreeze wrote: post #76

I am thinking when was the last time GitLab went down, I rarely or did not see downtime experience like this bad with Gitlab, I was seeing history, https://status.gitlab.com/pages/history/ They do have some latency or slowness issues, but couldn't find like whole system down thing, Like in one of the comments here, reminded me of 2017 incident, https://about.gitlab.com/blog/2017/02/10/postmortem-of-datab... They shou…

That GitLab downtime happened when we had deadlines, luckily git isn't a centralised platform so we merged our changes on a new GitHub repo we created.

Also the GitLab sluggishness reminds me of their daemon which kills the server to control memory leaks[1], although this probably isn't the main cause of the platform's slowness.

[1]: https://about.gitlab.com/blog/2015/06/05/how-gitlab-uses-uni...

Re: GitHub was down

#112

Post by Symbiote »

aaomidi wrote: post #69
haik90 wrote: post #23

Earlier quoted context omitted.

after this downtime today. We're finally start discussing (again) to use our own Gitlab.

I love when people suggest this and when someone asks okay who do we put in charge of having better availability than github no one has a good answer.

In your case, maybe the same people that keep microsoft.com online.

Re: GitHub was down

#113

Post by HelloNurse »

gregoriol wrote: post #32
rvz wrote: post #13

14 Days ago, they went down [0]. And today it's happening again. Twice in less than a month. Another reminder to self host via solutions like GitLab or Gitea. [1] [0] https://news.ycombinator.com/item?id=23675864 [1] https://news.ycombinator.com/item?id=23676072

Could you provide details on how you plan to be more reliable with a self-hosted solution? what kind of archtecture would you use? how many people will be involved in maintaining?

No random high-risk infrastructure updates?

A server is more likely to go belly-up by accident than the presumably redundant GitHub infrastructure, but its life is much less interesting.

Re: GitHub was down

#114

Post by jaekash »

joelhaasnoot wrote: post #105
jaekash wrote: post #88

Earlier quoted context omitted.

> We've been self hosting GitHub Enterprise for years and it has had no downtime other than during scheduled updates (at night, when nobody's at work). Is it free to self host like GitLab or Gitea? If not, what does it offer that you can't get from those?

On-premise or own-cloud hosting is quite import for some customers.

> On-premise or own-cloud hosting is quite import for some customers.

Yes it is, and GitLab and Gitea both offer that, which brings us back to the question I asked, what does GitHub Enterprise offer that you can't get from GitLab and Gitea?

Re: GitHub was down

#115

Post by jaekash »

echelon wrote: post #99
jaekash wrote: post #86

Earlier quoted context omitted.

Why are you not using an alternative?

> Why are you not using an alternative? I don't need two CI/CD pipelines. Nor do I have have time to build something like that. I need to spend my time working on the core product. The risk here is that my deploy SLA is tied to Github. I accepted this risk as the cadence of my deploys tends to be once to twice a day on average and Github is usually available (many nines). I made a choice, and now that choice is bitin…

> I don't need two CI/CD pipelines.

Maybe my question was unclear, but I was not asking why are no not using two services, I was asking why are you not using an alternative, and I meant instead of GitHub, not in addition to GitHub.

> If this is the last of it and Microsoft makes a commitment to not break things, I'll likely stay as the effort to move is nonzero.

Commitment without liability is not a commitment. It is just empty words. And I don't see Microsoft making any commitments with liability on their part for a free service.

Re: GitHub was down

#116

Post by creshal »

gregoriol wrote: post #32
rvz wrote: post #13

14 Days ago, they went down [0]. And today it's happening again. Twice in less than a month. Another reminder to self host via solutions like GitLab or Gitea. [1] [0] https://news.ycombinator.com/item?id=23675864 [1] https://news.ycombinator.com/item?id=23676072

Could you provide details on how you plan to be more reliable with a self-hosted solution? what kind of archtecture would you use? how many people will be involved in maintaining?

Git is decentralized by default, so all you really need is for your self-hosted version to not crash at the same time as github does. That's not too hard to pull off.

Re: GitHub was down

#117

Post by jaekash »

majkinetor wrote: post #108
jaekash wrote: post #81

Earlier quoted context omitted.

> It's just your own feelings. Nope. Github is down, I don't feel it is down, it is down. Office 365 falls apart when two people try edit a spreadsheet at once. Outlook does not allow you to unreject a meeting invite. Windows won't report DLL errors over a powershell terminal, Azure takes 10 minutes to delete a VM instance, Azure DevOps is so poorly designed that nobody can figure out how to find a repo without someo…

You seem to have problems with online stuff. I don't use cloud and have no such problems.

I prefer my option which is to just not use MS products but to each their own.

Re: GitHub was down

#118

Post by kenhwang »

hnlmorg wrote: post #87
kenhwang wrote: post #28

Earlier quoted context omitted.

COVID related work from home adjustments is my guess.

That might be the trigger but it wouldn't be the root cause. For example web and backend servers for cloud services shouldn't be affected by whether people are sat in one location or another. However if those systems requires lots of maintenance to keep running and people are less available due to COVID then you'd see a rise in downtime. But that would mean it's not COVID that's the problem, it's the amount of mainte…

Doesn't have to be on service side either. Could be that user patterns are changing too, testing scenarios that aren't well worn on GitHub.

Re: GitHub was down

#119

Post by victords »

sairamkunala wrote: post #50
Illniyar wrote: post #15

Seems like things started to go down the drain somewhere around February 2020. https://www.githubstatus.com/uptime/kr09ddfgbfsf?page=2 Wonder what was the trigger for the reliability hit - actions went GA on nov 2019, so it's something else (or possibly a combination of things)

They started doing availability reports. Las months details in the blog post below with summary of the issue. Stay tuned till next month for the current outage. https://github.blog/2020-07-08-introducing-the-github-availa...

Nice, I did not know they had that!

Is there a way to subscribe to this? Did not find an option for the reports neither at the Status page or at the Blog page.

Re: GitHub was down

#120

Post by anemic »

Protip:

Always have an extra customer, like the flowershop downstairs. Let her borrow your wifi in exchange for some office flowers. Now she is technically your customer.

When your shit goes down and nothing works you can still write "some of our customers are experiencing issues" in the statuspage as the flowershop still has wifi (hopefully).

Post reply on HN