Live data from Hacker News

Incident with Github.com [resolved]

githubstatus.com

371–380 of 1001 posts

Re: Incident with Github.com [resolved]

#371
post #124
post #29

I had a lot of goodwill for GitHub but I think today is the tipping point. Looking at a unicorn page, I feel this lingering hope that it's transient (like it usually was in the old days) but my mind reassures me it's probably going to be a long full outage again. The hope is dead.

They should replace the unicorn image with a horse or at least a zebra during ongoing incidents.

Donkey, surely.

Re: Incident with Github.com [resolved]

#373

Earlier quoted context omitted.

Boomer opinion but trusting third parties to be critical infrastructure, especially with no SLA in sight, will always end in tears. "The cloud" is very convenient, but its providers will never care about your infrastructure or your customers as much as you will.

3P-maintained infrastructure is what makes civilizations work efficiently. We're not all digging our own wells, generating our own electricity, and burning or burying our own garbage.

You are still supposed to be prepared for outages.

Re: Incident with Github.com [resolved]

#374
post #256

Earlier quoted context omitted.

Boomer opinion but trusting third parties to be critical infrastructure, especially with no SLA in sight, will always end in tears. "The cloud" is very convenient, but its providers will never care about your infrastructure or your customers as much as you will.

Boomer here as well, but I'd add that trusting your own org for critical infra usually also ends in tears. Most everything in IT involves failure, including in well designed systems designed by great engineers. I worked for a few years in an exceedingly well capitalised place which ran everything in their own data centers, money no object, with a truck parked somewhere, ready to go, with a smaller version of our crit…

At this point github is barely managing one nine.

Re: Incident with Github.com [resolved]

#375

Earlier quoted context omitted.

That's an overly optimistic view of things. For a lot of us GitHub is critical infrastructure, which if it goes down loses us and our customers money.

How is github being down losing you guys money?

I was in the middle of a hot fix. Our pipeline goes through Github.

Re: Incident with Github.com [resolved]

#376
post #99

This is starting to feel like Twitter. For a variety of reasons, we had a centralized place where everyone of a particular set of persuasions could connect, and this had outsize benefits for the community as a whole. That place is becoming untenable, and with the loss of goodwill and stability, the community’s started to dissipate. But there isn’t one obvious transition candidate, so the diaspora is finding itself sp…

Luckily we already have a distributed version control system that can take over github: https://fossil-scm.org/home/doc/trunk/www/index.wiki I suppose people can use git too if they ever figure out how to send emails from a dedicated client.

I like Fossil, but there are good reasons to use git too.

There are many git hosting options, including some for self hosting. its a lot easier to switch to a different version control host than to a different version control system.

Re: Incident with Github.com [resolved]

#377

To everyone who is angry: calm down. Github’s servers are constantly on fire as their usage increased something like 50x due to LLM sloppers pushing large amounts of trash code Unless you have a emergency hotfix (you don’t), go hit the gym or walk outside. If a work tool going down triggers you enough mentally to start angrily ranting online, it’s a sign you need to chill out and focus more on your health

> constantly on fire as their usage increased something like 50x due to LLM sloppers pushing large amounts of trash code

I find it hard to believe that github APIs don't have rate limits to handle traffic spikes.

Re: Incident with Github.com [resolved]

#378

Earlier quoted context omitted.

Luckily we already have a distributed version control system that can take over github: https://fossil-scm.org/home/doc/trunk/www/index.wiki I suppose people can use git too if they ever figure out how to send emails from a dedicated client.

> Luckily we already have a distributed version control system that can take over github Surprised that this wasn't just a link to Git. The biggest benefit of Git over predecessors was that it enabled decentralization. Github re-centralized the common case for the sake of convenience.

> "Crypto had attracted grifters from the beginning, even though its most respected figures were more interested in changing the world. As Silicon Valley investors piled in after them, they wanted to fund companies that would become essential to the crypto economy, allowing them to make vast amounts of money. Entrepreneur after entrepreneur discovered that the best way to turn a profit in a decentralized economy was to figure out ways to centralize parts of it again." (Henry Farrell, Underground Empire)

Re: Incident with Github.com [resolved]

#379
post #92

Earlier quoted context omitted.

Not the same company but similar discussions here: - It was believed that GitHub would be more reliable than our self hosted Gitlab CI runners. This has proven to be very false. - It was believed it would be easier to write tooling. I’m not on the dev tooling team, so can’t confirm if they found it true or not, but from the outside I see a lot more “Github can’t do that” than I did “Gitlab can’t do it”

Those are both hilarious to me. In my last FTE role we had the exact opposite beliefs - we wanted to leave GitLabs increases in pricing and move to annual sub, but the reliability of self-hosted runners and the flexibility (and clarity) of the CI system was fantastic.

> Those are both hilarious to me.

Why is it hilarious to believe that the industry leader might be better at something than doing it yourself?

Re: Incident with Github.com [resolved]

#380

If you are required to use a Microsoft tool is Azure DevOps now a better choice? Our Azure hosted instance rarely (if ever) has had an outage that effects our ability to work.

The worry I would have is, is anybody at MS still working on it? It is barely updated anymore: https://azure.microsoft.com/en-us/updates?filters=%5B%22Azur... That's not necessarily a bad thing for stability, but long-term I would not be surprised if it gets EOL-ed, or radically shrunk to just a basic Git mirroring service.

I'm not sure that link is a fair representation of the updates. This has more detail https://learn.microsoft.com/en-gb/azure/devops/release-notes...

Discussion about it going ELO has been around since 2020-ish. But that is a worry i agree.

I get the feeling there is a long tail of old-school corps (migrated from the TFS days) that will keep the lights on. And for some orgs you just need "good enough" CI/CD pipelines that are reliable.

Maybe I'm a weirdo, but i find the kanban/sprint board pretty usable as well.

Post reply on HN