Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GitHub appears to be struggling with measly three nines availability

theregister.com

Post reply on HN
Print view View original

111–120 of 255 posts

Re: GitHub appears to be struggling with measly three nines availability

#111

Post by hobofan »

derfniw wrote: post #104
lijok wrote: post #100

Earlier quoted context omitted.

The problem is actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 probably doesn’t do this same pinning, and the actions ecosystem is such an intertwined mess that any single compromised action can propagate to the rest

Well, it is a git commit hash of the action repo that contains the transpiled/bundled javascript. Like: https://github.com/actions/checkout/tree/11bd71901bbe5b1630c... So I'm pretty sure that for the same commit hash, I'll be executing the same content.

This is true specifically for actions/checkout, but composite actions can have other actions as dependencies, and unless the composite action pins the versions of its dependencies, it is vulnerable for this attack.

This article[0] gives a good overview of the challenges, and also has a link to a concrete attack where this was exploited.

[0]: https://nesbitt.io/2025/12/06/github-actions-package-manager...

Re: GitHub appears to be struggling with measly three nines availability

#113

Post by 827a »

I don't want to give too much credit to Github, because their uptime is truly horrendous and they need to fix it. But: I've felt like its a little unfair to judge the uptime of company platforms like this; by saying "if any feature at all is down, its all down" and then translating that into 9s for the platform.

I never use Github Copilot; it does go down a lot, if their status page is to be believed; I don't really care when it goes down, because it going down doesn't bring down the rest of Github. I care about Github's uptime ignoring Copilot. Everyone's slice of what they care about is a little different, so the only correct way to speak on Github's uptime is to be precise and probably focus on a lot of the core stuff that tons of people care about and that's been struggling lately: Core git operations, website functionality, api access, actions, etc.

Re: GitHub appears to be struggling with measly three nines availability

#114

Post by rtaylorgarlock »

imglorp wrote: post #96
cedws wrote: post #18

While GitHub obsess over shoving AI into everything, the rest of the platform is genuinely crumbling and its security flaws are being abused to cause massive damage. Last week Aqua Security was breached and a few repositories it owns were infected. The threat actors abused widespread use of mutable references in GitHub Actions, which the community has been screaming about for years, to infect potentially thousands of…

Public service announcement You can pin actions versions to their hash. Some might say this is a best practice for now. It looks like this, where the comment says where the hash is supposed to point. Old --> uses: actions/checkout@v4 New --> uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4 There is a tool to sweep through your repo and automate this: https://github.com/mheap/pin-github-action

My preferred tool to solve these issues is called 'gitlab'

Re: GitHub appears to be struggling with measly three nines availability

#115

Post by dijit »

tpmoney wrote: post #108
dijit wrote: post #28

I’m surprised it’s even as high as three nines, at one point in 2025 it was below 90%; not even a single nine.[0] (which, to be fair includes co-pilot, which is the worst of availabilities). People on lobsters a month ago were congratulating Github on achieving a single nine of uptime.[1] I make jokes about putting all our eggs in one basket under the guise of “nobody got fired for buying x; but there are sure a lot…

> These two things can’t be true simultaneously Sure they can. Perhaps a useful example of something like this would be to consider cryptography. Crypto is ridiculously complex and difficult to do correctly. Most individual developers have no hope of producing good cryptographic code on the same scale and dependability of the big crypto libraries and organizations. At the same time these central libraries and organiz…

That’s an excellent example. OpenSSL, by virtue of trying to do everything is the most buggy implementation of TLS generally available today leading to the point where there have been hard forks designed to reduce the scope to limit this damage.

I’d go so far as to say that there are more crypto libraries than there are “default” options for SaaS Git VCS (Gitlab and Github are the mainstay in companies and maybe Azure Devops if you hate your staff- nobody sensible is using bitbucket) but for TLS implementations there’s RustTLS, GnuTLS, BoringSSL, LibreSSL, WolfSSL, NSS, and AWS-LC that come to mind immediately.

Re: GitHub appears to be struggling with measly three nines availability

#116

Post by righthand »

rtaylorgarlock wrote: post #114
imglorp wrote: post #96

Earlier quoted context omitted.

Public service announcement You can pin actions versions to their hash. Some might say this is a best practice for now. It looks like this, where the comment says where the hash is supposed to point. Old --> uses: actions/checkout@v4 New --> uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4 There is a tool to sweep through your repo and automate this: https://github.com/mheap/pin-github-action

My preferred tool to solve these issues is called 'gitlab'

CircleCI

TravisCI

Jenkins

scripts dir

Etc

Re: GitHub appears to be struggling with measly three nines availability

#117

Post by lucideer »

827a wrote: post #113

I don't want to give too much credit to Github, because their uptime is truly horrendous and they need to fix it. But: I've felt like its a little unfair to judge the uptime of company platforms like this; by saying "if any feature at all is down, its all down" and then translating that into 9s for the platform. I never use Github Copilot; it does go down a lot, if their status page is to be believed; I don't really…

You're right that labelling any outage as "Github is down" is an overgeneralisation, & we should focus on bottlenecks that impact teams in a time sensitive matter, but that isn't the case here. Their most stable service (API) has only two 9s (99.69%).

They're not even struggling to get their average to three 9s, they're struggling to get ANY service to three 9s. They're struggling to get many services to two 9s.

Copilot may be the least stable at one 9, but the services I would consider most critical (Git & Actions) are also at one 9.

Re: GitHub appears to be struggling with measly three nines availability

#118

Post by DailyGeo »

The availability expectations gap is interesting from an education standpoint. Students are tought that 99.9% sounds impressive without contextualizing what that means in practice — roughly 8 hours of downtime per year. For a platform that millions of developers depend on as critical infrastructure during work hours, that math hits very differently than it does for a consumer app.

Re: GitHub appears to be struggling with measly three nines availability

#119

Post by calvinmorrison »

lucideer wrote: post #117
827a wrote: post #113

I don't want to give too much credit to Github, because their uptime is truly horrendous and they need to fix it. But: I've felt like its a little unfair to judge the uptime of company platforms like this; by saying "if any feature at all is down, its all down" and then translating that into 9s for the platform. I never use Github Copilot; it does go down a lot, if their status page is to be believed; I don't really…

You're right that labelling any outage as "Github is down" is an overgeneralisation, & we should focus on bottlenecks that impact teams in a time sensitive matter, but that isn't the case here. Their most stable service (API) has only two 9s (99.69%). They're not even struggling to get their average to three 9s, they're struggling to get ANY service to three 9s. They're struggling to get many services to two 9s. Copi…

ONLY TWO NINES! Meanwhile vital government services here have a whopping 25% availability.
Post reply on HN