61–70 of 176 posts
> We've identified an issue with a database primary and are failing over to a replica immediately This is why it's hard to take GitHub seriously. How can a single database cause an outage for everyone? This is amateur stuff. Have they no sharding or partitioning internally? Paying customers should not be impacted in the same way as free ones are.
Earlier quoted context omitted.
Where do you see that? Common Name (CN) www.dayswithoutgithubincident.com Organization (O) Common Name (CN) YR1 Organization (O) Let's Encrypt Issued On Monday, August 10, 2026 at 10:01:51 AM Expires On Sunday, November 8, 2026 at 9:01:50 AM
also seeing an expired cert: Common Name R12 Validity Not Before Tue, 10 Feb 2026 17:28:52 GMT Not After Mon, 11 May 2026 17:28:51 GMT
Earlier quoted context omitted.
I wonder what is this database, and why it is hard to fall-over automatically.
RDBMS replication and failover is way more difficult and manual than anyone would like. You can't just set up two postgres, tell them they're clustered and have it basically work; at a minimum you have to design the client to somehow know which one is currently the master, or use some sort of proxy (which becomes its own SPOF). RDBMS integrity basically requires that one master server is responsible for the whole dat…
Earlier quoted context omitted.
I wonder what is this database, and why it is hard to fall-over automatically.
Possibly vitess from the latest update: > primary failover briefly improved performance but did not fully mitigate, we've throttled inbound traffic and are investigating upstream Vitess issues
GitHub needs to completely bifurcate their enterprise/paid services from their free services at the infra level.
They have that-ish as an option: https://docs.github.com/en/enterprise-cloud@latest/admin/dat... I'm told that GitHub has asserted to us that moving to this model means we would not be exposed to github.com outages. It's not at feature parity with github.com though.
We're currently on "GitHub Enterprise Cloud" on github.com and are affected by this outage (even though we use self-hosted runners!), but we're not on "GitHub Enterprise Cloud with data residency" on *.ghe.com, which I understand is/may not be affected by this outage?
GitHub needs to completely bifurcate their enterprise/paid services from their free services at the infra level.
Fun read about Azure and having 173 agents running a node: https://isolveproblems.substack.com/p/how-microsoft-vaporize... Probably just a coincidence that Github started to have issues after beginning their move to Azure at the end of last year.
>Update - primary failover briefly improved performance but did not fully mitigate, we've throttled inbound traffic and are investigating upstream Vitess issues And now they're blaming their upstream vendor! Embarrassing stuff to be writing on a public page.