Live data from Hacker News

Disruption with Some GitHub Services – Resolved

githubstatus.com

31–40 of 176 posts

Re: Disruption with Some GitHub Services – Resolved

#31
>Update - primary failover briefly improved performance but did not fully mitigate, we've throttled inbound traffic and are investigating upstream Vitess issues

And now they're blaming their upstream vendor! Embarrassing stuff to be writing on a public page.

Re: Disruption with Some GitHub Services – Resolved

#32
post #10

GitHub needs to completely bifurcate their enterprise/paid services from their free services at the infra level.

enterprise is mostly separate, is it not? uptimes are significantly more reasonable on the enterprise status pages

We are in GHEC right now and GitHub Actions is not working. It's been down every time githubstatus.com says it's down.

Re: Disruption with Some GitHub Services – Resolved

#34

> We've identified an issue with a database primary and are failing over to a replica immediately This is why it's hard to take GitHub seriously. How can a single database cause an outage for everyone? This is amateur stuff. Have they no sharding or partitioning internally? Paying customers should not be impacted in the same way as free ones are.

I wonder what is this database, and why it is hard to fall-over automatically.

Possibly vitess from the latest update:

> primary failover briefly improved performance but did not fully mitigate, we've throttled inbound traffic and are investigating upstream Vitess issues

Re: Disruption with Some GitHub Services – Resolved

#36
Things can go wrong, but really, its been a lot and we're normalizing that to an unhealthy degree...

I wonder if it was down that much, if users would get credits the way we pay when we use the services - its kind of ridiculous for a critical service to be down that much and all we do is "ah okay, its just github". Like, as if that was normal to be down that much...

Re: Disruption with Some GitHub Services – Resolved

#39

>Update - primary failover briefly improved performance but did not fully mitigate, we've throttled inbound traffic and are investigating upstream Vitess issues And now they're blaming their upstream vendor! Embarrassing stuff to be writing on a public page.

I read that as an upstream service they own, but I agree the wording a bit weird.

Re: Disruption with Some GitHub Services – Resolved

#40

>Update - primary failover briefly improved performance but did not fully mitigate, we've throttled inbound traffic and are investigating upstream Vitess issues And now they're blaming their upstream vendor! Embarrassing stuff to be writing on a public page.

Why is that a problem if it _is_ an upstream vendor problem? (assuming it is)
Post reply on HN