Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GitHub Git Operations Are Down

githubstatus.com

Post reply on HN
Print view View original

121–130 of 271 posts

Re: GitHub Git Operations Are Down

#121

Post by jcims »

parpfish wrote: post #100
jcims wrote: post #88

Tangential at best but I work in a regulated industry and we're starting to get some heavy scrutiny from external auditors and regulators on the way we plan to address extended outages and forced exits from third party vendors. This goes beyond basic DR/BCP and plans are reviewed with at least a high level scenario exercise. https://www.federalregister.gov/documents/2023/06/09/2023-12... On the surface a product like…

but if you self host, how do you anticipate power outages and natural disasters? you shift it from a problem of software reliability to a problem of physical infrastructure. at some point in the chain somebody has to do that, but i'd prefer the person doing that was somebody with DEEP experience in it that could give you some nice confident assurances.

Most of the companies exposed to this are already going to be running a number of geographically diverse datacenters and have been exposed to regulations around basic DR/BCP for decades.

>at some point in the chain somebody has to do that, but i'd prefer the person doing that was somebody with DEEP experience in it that could give you some nice confident assurances.

Yes. This is part of the reason why services like git are being moved outside the datacenter. Most of the product offerings on the market don't scale well, have terrible reliability and are still very expensive to run.

Re: GitHub Git Operations Are Down

#122

Post by arrowsmith »

__MatrixMan__ wrote: post #119
School-Cotton wrote: post #111

Earlier quoted context omitted.

No, they'll continue to feel like that even after that. GitHub being down once in a blue moon is more acceptable to the vast majority of users than having to cobble together your own nerdy distributed version of everything.

I was imagining something a bit more disastrous than that. A big enough solar flare could take parts of the planet offline for months. Years if they can't source enough replacement transformers. There are also political reasons that countries go offline. Then it'll be up to the nerds who manage to cobble together their own distributed version of everything--even if it's a significantly reduced definition of everythin…

If large parts of the planet lose their digital infrastructure for months, I really don’t think that “finding a good platform to host my code” is going to be one of my biggest problems.

Re: GitHub Git Operations Are Down

#123

Post by FridgeSeal »

tom1337 wrote: post #48

Not sure if I am just noticing GitHub's issues more often as I am using their tools pretty much every day but their availability is kinda not great. Be it Actions failing or something "not core business" (read: git operations) but I can't remember a month in the past where I was not annoyed by any outage on GitHub

The whole product feels like it’s getting g progressively jankier.

The front-end is glacial nowadays and frequently has issues actually loading the page, actions frequently has some kind of panic attack and breaks or just grinds along a glacial speeds. The UX has gotten worse (why no merge-queue button?).

Re: GitHub Git Operations Are Down

#124

Post by evbogue »

crest wrote: post #13

It's almost as if only Git is distributed, but people sold out to Github for convenience. Too bad Git lacks a distributed bug tracker and wiki system like Fossil. Guess Github has to fail a lot more for things to change.

Makes me wish for the good old days of git-ssb

Re: GitHub Git Operations Are Down

#125

Post by liontwist »

bastardoperator wrote: post #83
crest wrote: post #13

It's almost as if only Git is distributed, but people sold out to Github for convenience. Too bad Git lacks a distributed bug tracker and wiki system like Fossil. Guess Github has to fail a lot more for things to change.

It's this type of negativity ruining the internet. Nothing thoughtful to say, nothing to add, and hoping for failure. I hope everything is okay over there...

They are advocating for decentralized tools. That’s actionable and hoping to prevent failure.

Re: GitHub Git Operations Are Down

#126

Post by jcims »

macintux wrote: post #118
jcims wrote: post #88

Tangential at best but I work in a regulated industry and we're starting to get some heavy scrutiny from external auditors and regulators on the way we plan to address extended outages and forced exits from third party vendors. This goes beyond basic DR/BCP and plans are reviewed with at least a high level scenario exercise. https://www.federalregister.gov/documents/2023/06/09/2023-12... On the surface a product like…

The nice thing about git, from my perspective, is that if your entire hosted service vanishes, you can still reconstruct what you need from your users’ working directories. All of the important branches should be there. Somewhere. And any important integration branches that aren’t cached can be reconstructed. Of all the many dependencies on cloud services, git is by far the last I’d worry overly much about.

I think my point is we are losing the option to worry about it or not. There needs to be an answer with a runbook to restore services within a given window of time should we lose that vendor.

Re: GitHub Git Operations Are Down

#127

Post by pnw_throwaway »

0xbadcafebee wrote: post #78

> all Git operations were unavailable due to a configuration change eats popcorn waiting for explanation of why they didn't catch it in non-prod

Have you never fixed a prod-specific bug before? They’re common enough, and it sounds like they patched this one within a reasonable amount of time.

Re: GitHub Git Operations Are Down

#129

Post by ascar »

__MatrixMan__ wrote: post #119
School-Cotton wrote: post #111

Earlier quoted context omitted.

No, they'll continue to feel like that even after that. GitHub being down once in a blue moon is more acceptable to the vast majority of users than having to cobble together your own nerdy distributed version of everything.

I was imagining something a bit more disastrous than that. A big enough solar flare could take parts of the planet offline for months. Years if they can't source enough replacement transformers. There are also political reasons that countries go offline. Then it'll be up to the nerds who manage to cobble together their own distributed version of everything--even if it's a significantly reduced definition of everythin…

Even if that's a concern in a doomsday scenario, self-hosting gitlab is super easy and a good (some would argue better) alternative.

Re: GitHub Git Operations Are Down

#130

Post by ajb »

sixo wrote: post #19
crest wrote: post #13

It's almost as if only Git is distributed, but people sold out to Github for convenience. Too bad Git lacks a distributed bug tracker and wiki system like Fossil. Guess Github has to fail a lot more for things to change.

people could not be more clear that their preference is for reliable and easy-to-use centralized services maintained by professionals, and not decentralized systems that require a great deal of user expertise

There is not actually a need to choose between a single centralised monopoly and going full-on techno-prepper and running all your own services in your garage as an individual . We could have intermediate points, such as having services run by professionals but based on portable open standards.
Post reply on HN