Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GitHub Outage Tracker: Is GitHub Cooked?

isgithubcooked.com

Post reply on HN
Print view View original

111–120 of 195 posts

Re: GitHub Outage Tracker: Is GitHub Cooked?

#111

Post by pocksuppet »

olyjohn wrote: post #103
majormajor wrote: post #81

Earlier quoted context omitted.

Random AI slop code pushers don't pay the bills. Traffic isn't revenue. I've started suggesting to my CTO that we move to something else because we do pay the Github bills and yet we are also affected by other people's activity . Avoiding that is not a hard engineering problem for an alternative. A world where a record amount of code is pushed to GH but a large amount of paying customers leave would be "cooked"

Yeah I don't feel bad for Microsoft, but I don't feel bad for people who keep paying them for it either. This has been going on forever now. Nothing will happen until people start leaving. I don't honestly understand why so much shit is on GitHub when there are so many alternatives, anyways. It's fucking git! Can't people do or try anything different, ever?

People are on GitHub because it's the cool thing to do, and there is no concept of anything else that could matter.

Re: GitHub Outage Tracker: Is GitHub Cooked?

#112

Post by pocksuppet »

rvz wrote: post #20

They were cooked the moment they got acquired by Microsoft. This is why I foresaw that centralizing everything to GitHub was just generally a bad idea 6 years ago. [0] Now that there is no CEO of GitHub, there is no point to GitHub improving. [0] https://news.ycombinator.com/item?id=22867803

6 years is a good run. For comparison, every 6 years there's a not insignificant chance you die. A restaurant that lasted 6 years would be an outstanding success. 6 years ago the very first LLMs were a novelty toy. 6 years before that was just after HTTPS became popular. 6 years before that was about when the iPhone came out.

Re: GitHub Outage Tracker: Is GitHub Cooked?

#113

Post by bmitc »

kashnote wrote: post #12

I think we need to have a little more sympathy for GitHub. You could justify the jabs when we could all blame any outage on the migration to Azure, but then they shared numbers around the scale they're dealing with now that everyone is constantly building and pushing with AI. I think it's commendable that they're not limiting access to the site or (intentionally) throttling newcomers. Yes, they need to get this figur…

GitHub literally went all in on doing almost nothing but AI-related feature additions for Copilot. So they built their own prison.

I don't have sympathy for them. They have disrupted my work so much in the past month that it's ridiculous, and my company likely pays them millions of dollars. And you can see from this site's plotting that there is a trend towards more frequent and more critical outages. It has been extremely bad the past month.

Re: GitHub Outage Tracker: Is GitHub Cooked?

#114

Post by robcohen »

chris_money202 wrote: post #104
cr125rider wrote: post #91

Earlier quoted context omitted.

That seems a bit crass but the underlying sentiment stands. Microsoft has more money than God. 4T valuation. When you say you’re worth that much, no excuses. Figure it out.

So your proposal is for Microsoft to invest their free cash in helping support a bunch of AI coders maintain their pet projects? Sounds like you should be a CEO!

Start charging more, cut down on CI, figure it out. You don't NEED to give away so much if it means your service is going down.

Re: GitHub Outage Tracker: Is GitHub Cooked?

#115

Post by pydry »

pixelatedindex wrote: post #99
logicchains wrote: post #85

Earlier quoted context omitted.

How low would uptime have to get before you'd consider holding the engineers personally responsible for it?

Why are we holding the engineers accountable and not the CEO? The company isn’t one department. Responsibility should bubble up to the top.

They dont have a CEO ever since their last CEO said that human programming "wasn't going anywhere".

Re: GitHub Outage Tracker: Is GitHub Cooked?

#116

Post by atherton94027 »

tyre wrote: post #58
gogobio wrote: post #23

Earlier quoted context omitted.

Sympathy? It's a Microsoft company that is being ran with a consistency of a startup in early seed rounds. Their downtime is abhorrent and unacceptable as far as enterprise goes. Their engineers look like absolute amateurs allowing for such low class work it results in their customers experiencing industry leading downtime.

This is uncharitable, rude, and pretty baseless. Unless you have a lot of direct, personal information about GitHub engineers, they’re dealing with a huge spike in traffic. Is their uptime acceptable? No. But personal attacks aren’t necessary or constructive.

Please, read between the lines. Github actions has been a mess since 2019 at least. None of the instability is new, before this unprecedented growth (for a service that's supposed to scale horizontally) the excuse du jour was the azure migration, before that it was the high rate of shipping post acquisition.

Core parts of the product, like navigating to individual files in a code review, are broken

Re: GitHub Outage Tracker: Is GitHub Cooked?

#117

Post by chris_money202 »

robcohen wrote: post #114
chris_money202 wrote: post #104

Earlier quoted context omitted.

So your proposal is for Microsoft to invest their free cash in helping support a bunch of AI coders maintain their pet projects? Sounds like you should be a CEO!

Start charging more, cut down on CI, figure it out. You don't NEED to give away so much if it means your service is going down.

There isn’t mission critical work being done on GH. You start charging more and the people in charge of finance at companies paying for GH will actually start paying attention. Those finance people don’t care rn because GH is affordable and helps get stuff done. There’s a threshold that exist at every company and once it’s crossed the people who can make money decisions start asking questions

Re: GitHub Outage Tracker: Is GitHub Cooked?

#118

Post by vel0city »

jjice wrote: post #17
herpdyderp wrote: post #15

Earlier quoted context omitted.

> I think it's commendable that they're not limiting access to the site or (intentionally) throttling newcomers. I don't think this is commendable at all. I give GitHub a lot of money and I'm tired of it being wasted with downtime.

It's a shame that the GitHub org that we use at my job that we pay a lot of money for gets affected the same way my personal nonsense does. I don't know the architecture or any of that, but I feel like there could be (and it's not like they would've really known this until the last year or two with the massive spike) separate infrastructure for paid users/orgs vs free the same way they make the distinction with enter…

You can pay for GitHub Enterprise Cloud which is essentially their private GitHub deployment stack that they'll host for you. It's theoretically isolated, supposed to be completely separate from these outages.

It's a lot more expensive and has a good bit more limitations to the regular SaaS product.

Re: GitHub Outage Tracker: Is GitHub Cooked?

#119

Post by Freedom2 »

vrosas wrote: post #90
logicchains wrote: post #85

Earlier quoted context omitted.

How low would uptime have to get before you'd consider holding the engineers personally responsible for it?

Agreed. At some point to have to stop hugging your ops team and start firing them.

Agreed. In fact, keep firing ops until someone gets it done. There's no way your company will build a reputation for not supporting their teams.

Re: GitHub Outage Tracker: Is GitHub Cooked?

#120

Post by Freedom2 »

californical wrote: post #61
zvmaz wrote: post #56

Earlier quoted context omitted.

> Their engineers look like absolute amateurs allowing for such low class work it results in their customers experiencing industry leading downtime. I gather that you have intimate and deep knowledge on the teams and the problems they try to solve there.

If I own a restaurant and buy bread from a supplier — BreadHub. And 99% of the bread that I get is good but 1% of the loaves, they forgot to add flour. Consistently, for years, they always have loaves missing a key ingredient that I still end up paying for. I can be pretty sure that BreadHub have a pretty major internal issue, and should probably be questioning their competence, regardless of their “scale”, and witho…

I don't think this is an apt analogy. The bread makeup is still there, as far as I'm aware, no users have lost any data or are missing "key" ingredients.

An outage is more like a shipping issue with the supplier, if it's owned wholly by them.

Post reply on HN