EDIT: sorry i meant this rant at the one complaining for the free service not for the paid customers (which is unacceptable)
Incident with Actions and Pages
141–150 of 407 posts
Re: Incident with Actions and Pages
#142Someone said GitHub is racing to the mythical "zero nines of availability" and I love it
Hmm... 88.8888888%? Jesus, that's both horrible and seems within reach.
Re: Incident with Actions and Pages
#143Earlier quoted context omitted.
Same. It's weird how I always find out that GitHub is down before GitHub does. Took 15 minutes before it appeared on githubstatus.com
More likely that 'update the Status site' lives a long way down their incident response plan, and they have alarms going off well before that
Re: Incident with Actions and Pages
#144Re: Incident with Actions and Pages
#145This is outrageous. Someone go create a Polymarket.
Re: Incident with Actions and Pages
#146free service is down again, let's everyone that use the service for free complain again!!! (sorry for the sarcastic comment but i find it crazy how people feel they are entitled when it's free) EDIT: sorry i meant this rant at the one complaining for the free service not for the paid customers (which is unacceptable)
Re: Incident with Actions and Pages
#147free service is down again, let's everyone that use the service for free complain again!!! (sorry for the sarcastic comment but i find it crazy how people feel they are entitled when it's free) EDIT: sorry i meant this rant at the one complaining for the free service not for the paid customers (which is unacceptable)
Re: Incident with Actions and Pages
#148Earlier quoted context omitted.
All these monitoring rules are of the format "when 500 errors > baseline for x minutes". Otherwise you'd have monitoring alerts every second. So it is normal for users to already see errors before github officially counts it as an outage.
In a high performance service with good maintenance and upkeep, you page for all 500s. A noisy pager forces the team to fix the 500s. Maybe the Github Actions infrastructure isn't run like that. edit: my oncall rotation notified on all 500s, 24/7, not just rates - https://news.ycombinator.com/item?id=48279262
Re: Incident with Actions and Pages
#149Re: Incident with Actions and Pages
#150For Git, all you technically need is ssh access and some backup strategy for your server. It would be bare bones but workable. And there are of course plenty of OSS things that are a lot nicer than that.
I'm still using gh and gh actions and we are mostly below the freemium layer with that. But it is kind of slow and honestly a dedicated vm plus some high CPU/memory workers we can spin up on a need to have basis might be a lot faster. With GH outages becoming more common, my hand might be forced a bit.
In recent weeks, I've spun up listmonk (mailing list solution), matrix (as a slack alternative), and a few other things specific to our software stack. A github alternative would be more of the same. We don't need a lot.
The main objection is that with more moving parts to worry about, the workload for me also increases. Things need updating, monitoring, backups, alerting (and responding to alerts), etc. That sucks up my time and that is scarce.
Another reason for self hosting these days is that with agentic AI tools, self hosted things are a lot easier to integrate into agentic systems. If it is self hosted, you don't have to worry about API limitations, rate limitations, walled gardens, etc. All the traditional SAAS silos are becoming a problem from that point of view. The more locked down it is, the bigger the motive for moving away from it. That's why we ditched Slack for Matrix. Slack is hopelessly locked down and tedious to deal with. Matrix is super easy for this.