using github in 2026 is a self own. its that simple at this point
GitHub Outage Tracker: Is GitHub Cooked?
121–130 of 195 posts
Re: GitHub Outage Tracker: Is GitHub Cooked?
#122Earlier quoted context omitted.
This is uncharitable, rude, and pretty baseless. Unless you have a lot of direct, personal information about GitHub engineers, they’re dealing with a huge spike in traffic. Is their uptime acceptable? No. But personal attacks aren’t necessary or constructive.
Please, read between the lines. Github actions has been a mess since 2019 at least. None of the instability is new, before this unprecedented growth (for a service that's supposed to scale horizontally) the excuse du jour was the azure migration, before that it was the high rate of shipping post acquisition. Core parts of the product, like navigating to individual files in a code review, are broken
> Core parts of the product, like navigating to individual files in a code review, are broken
I think this is a good argument to underline "It's not _just_ the scale". Adding to this, the Github Code Review experience is kind-of broken, the way comments/threads are stacked in the PR overview has not improved, pagination isn't really a thing, and these issues are age old. Hopefully, one day, Github will mature.
Re: GitHub Outage Tracker: Is GitHub Cooked?
#123What most people don't understand is that Microsoft can no longer engineer a technology product, they simply can't develop anything anymore. They can throw slop around, but they lack the corporate culture to actually build an innovative technology product. They can purchase other companies that have built products, but as soon as they do those products stop innovating and begin dying.
Re: GitHub Outage Tracker: Is GitHub Cooked?
#124What most people don't understand is that Microsoft can no longer engineer a technology product, they simply can't develop anything anymore. They can throw slop around, but they lack the corporate culture to actually build an innovative technology product. They can purchase other companies that have built products, but as soon as they do those products stop innovating and begin dying.
Re: GitHub Outage Tracker: Is GitHub Cooked?
#125I hear the comments that state we should have sympathy for github. For the individual ops/sre I can have that sympathy because it must a heck of a time internally to manage things. I can't however have sympathy for company itself. They have not set themselves or the community for success. I think it is the responsibility of the company to anticipate the issues or at least react in a way that would build trust. The fact is there is a systemic issue in the company that causes the repeated unreliableness and they have not rooted it out.
Re: GitHub Outage Tracker: Is GitHub Cooked?
#126I think we need to have a little more sympathy for GitHub. You could justify the jabs when we could all blame any outage on the migration to Azure, but then they shared numbers around the scale they're dealing with now that everyone is constantly building and pushing with AI. I think it's commendable that they're not limiting access to the site or (intentionally) throttling newcomers. Yes, they need to get this figur…
The question is, do we think Google would have the same problems running the service? (I'm biased from having worked there tho.)
Re: GitHub Outage Tracker: Is GitHub Cooked?
#127But after exploring some options for small but private and personal repos (not foss; not yet at least) I realised I am better off with a free Github private repo. There simply isn't a feasible free alternative for such a use-case without hitting, often a 50MB to say few 100s MB, the limit ASAP. There's gitlab.com, bitbucket (? haven't opened it in ages) but those are not better in any way for use cases like mine.
Are there any?
Re: GitHub Outage Tracker: Is GitHub Cooked?
#128There is alternatives for github, but all of them would be offline 100% is they got even 10% of traffic Github gets after AI.
Any dev can just have their own git mirror and work there, but replacing github is not really feasable, not one of the other services come even close.
If github is down for say 2h i dont really care. I can live with that. IF i did care i would just host my own.
Re: GitHub Outage Tracker: Is GitHub Cooked?
#129Earlier quoted context omitted.
Sympathy? It's a Microsoft company that is being ran with a consistency of a startup in early seed rounds. Their downtime is abhorrent and unacceptable as far as enterprise goes. Their engineers look like absolute amateurs allowing for such low class work it results in their customers experiencing industry leading downtime.
This is uncharitable, rude, and pretty baseless. Unless you have a lot of direct, personal information about GitHub engineers, they’re dealing with a huge spike in traffic. Is their uptime acceptable? No. But personal attacks aren’t necessary or constructive.
not constructive, but also not personal. rude but based on observed facts.
Re: GitHub Outage Tracker: Is GitHub Cooked?
#130Earlier quoted context omitted.
This is uncharitable, rude, and pretty baseless. Unless you have a lot of direct, personal information about GitHub engineers, they’re dealing with a huge spike in traffic. Is their uptime acceptable? No. But personal attacks aren’t necessary or constructive.
The phrasing could have been better, but we've talked as a team about cancelling a $100k/yr contract nearly half a dozen times the last year, and the only thing keeping us from doing so is various "compliance" issues, and to a lesser degree the friction from physically moving. That's a tenuous moat, and if I were a GH PM I'd probably want to know that the current instability is somewhere near critical mass.
For CI there are a number of drop-in commercially-available options. You can make it a staged migration, first trivially migrating the build runners, followed by the more complicated integration test and deployment runners. The CI harness itself can follow, and finally moving to a code hosting and visualization service is last.
The final step is challenging; likely the most difficult part is changing all the code references and imports. Shadowing changes would be straightforward. Training your likely 25-50 engineers to use the new code review UX would likely not take that long.
Considering the wasted engineering velocity during Github outages, it's worthwhile to do even a partial migration. Github's action runners have in my experience, been the most fragile part of the platform. Given the ease of moving build and merge queue runners to alternates, it's a no-brainer.