Good that the Billing functionality is still at 3 nines at least.
They got their priorities right for sure
Incident with Pull Requests, Issues, Git Operations and API Requests
181–190 of 221 posts
Re: Incident with Pull Requests, Issues, Git Operations and API Requests
#182New PR: revert GitHub software and infrastructure to version of June 1st, 2018. New PR: disable new user signups for 6 months HR initiative: all future KPIs automatically require three-nines availability; all bonuses are forfeited, regardless of accomplishments, if annual availability falls below target HR initiative: fire CEO and CTO
Imagine if we did this for countries.
Re: Incident with Pull Requests, Issues, Git Operations and API Requests
#183https://isgithubcooked.com Normally I defend GH in the comments of these incidents but it’s been an impressively bad month by their standards, even when you filter for critical components filter out sev-2’s and 3’s.
Name one thing Microsoft didn't run into the ground post-acquisition
Dave Cutler?
Re: Incident with Pull Requests, Issues, Git Operations and API Requests
#184Earlier quoted context omitted.
> It's not like writing English sentences is some huge secret you are sitting on that no one else knows. I'd actually say what really makes an excellent engineer stick out among many great engineers, is their ability to communicate clearly and knowing what needs to be communicated vs not, basically being way better at language and communication in general, and they also understand the important of it.
I agree. But I was talking about the "super secret" ability to write prompts, which pretty much anyone can do.
My point being that not everyone writes as good prompts as everyone else, the way you communicate, how clearly and how exact you are matters a lot, much more than you seemingly is under the impression of.
Same goes with the "LLM does web design" example from before, a web designer with great communication skills in web design, will (naturally) have a better prompt for something that'll potentially could look good, compared to a web designer that isn't at good at communicating what they actually want.
Re: Incident with Pull Requests, Issues, Git Operations and API Requests
#185Earlier quoted context omitted.
Why the heck would you want to do this. Using git as your undo chain sounds like a pretty awful thing to do.
I think it'd be pretty neat to be able to rebase my undo history on to a remote branch someone else is working on.
This is smart but also hilarious.
Re: Incident with Pull Requests, Issues, Git Operations and API Requests
#186Earlier quoted context omitted.
May has been filled with critical issues. It seems it's getting worse over time.
Commits are up 14x year-over-year https://x.com/kdaigle/status/2040164759836778878
They are highly responsible for all of that.
They are diversified a lot with a lot of random things instead of focusing on their core business. They have actively pushed people to use the service and feature more.
Think about countless actions that have to run almost at every push and PR push! Also, remember that we were used to use external services for "actions", and they basically killed the competition by offering their own CI actions at no cost to most users.
Also, they did a lot of reworks in the last years, not necessarily for the best like the PR diff page, and probably not in the most efficient way.
Re: Incident with Pull Requests, Issues, Git Operations and API Requests
#187Earlier quoted context omitted.
Of all the sites/graphs I've seen of GH outages, this one is the most striking IMO: https://damrnelson.github.io/github-historical-uptime/ Unfortunately, it doesn't look like it's being updated with new data. But it wouldn't look any better for GH if it was.
I wonder what the cause of this was? Microsoft Politics? Bureaucracy? Forced move to azure?
My guess would be a obnoxious and lethal mixture of all of the above.
Re: Incident with Pull Requests, Issues, Git Operations and API Requests
#188Earlier quoted context omitted.
Of all the sites/graphs I've seen of GH outages, this one is the most striking IMO: https://damrnelson.github.io/github-historical-uptime/ Unfortunately, it doesn't look like it's being updated with new data. But it wouldn't look any better for GH if it was.
FWIW, I'm not convinced that chart is necessarily an accurate representation of pre-acquisition reality. It would really surprise me if GitHub did not have a single sev-0 pre-acquisition, but it wouldn't surprise me if they were not formally captured and reported in a format that would make its way into their current status page's database.
Apparently, you aren't alone. :)
https://github.com/DaMrNelson/github-historical-uptime/issue...
Re: Incident with Pull Requests, Issues, Git Operations and API Requests
#189is it me or ever since AI coding became the norm, there have been way more outages with otherwise reliable services? I get downtime on Supabase every few weeks. Even Cloudflare. And now Github
Yes, because that caused the usage of the services to skyrocket, GitHub runs on Azure and Azure is experiencing capacity strain due to AI, so GitHub's services are struggling to auto-scale
GitHub’s data architecture (lots and lots of projects in a hierarchical distribution with immutable, highly transaction-based data elements) ought to be easily shardable to allow for massive scale-out. Especially a decade after acquisition by Microsoft. Even a major increase in activity (which was surely predicted internally years ago, given the marketing around AI) should be easily handled by a company with their resources. These repeated failures are indicative of massive mismanagement and misinvestment.
Re: Incident with Pull Requests, Issues, Git Operations and API Requests
#190This is getting ridiculous. One particularly concerning thing I’m seeing is that pull requests on both the web UI and API aren’t reflecting all commits or branch changes consistently. It would be very easy to merge something without realizing you’re not actually reviewing the full diff.
A taste of the agenticaly-developed world.
[flagged]