Earlier quoted context omitted.
That's fine, but rate limiting is a solved problem at this point. They need to rate-limit free accounts and be realistic about their limits instead of letting the system fail.
Free (and non-free) accounts are already rate limited. The likely problem is that individual accounts/repos rarely hit those limits, instead the explosive demand is due to a massive growth in the number of small, individual projects being creatd.
Incident with Github.com [resolved]
421–430 of 1001 posts
Re: Incident with Github.com [resolved]
#422We're moving to self hosted Gitea. It's never been easier to do devops with the help of LLMs. I cannot believe how flaky GitHub is. https://git.tinygrad.win/tinygrad/tinygrad
Re: Incident with Github.com [resolved]
#423Earlier quoted context omitted.
Luckily we already have a distributed version control system that can take over github: https://fossil-scm.org/home/doc/trunk/www/index.wiki I suppose people can use git too if they ever figure out how to send emails from a dedicated client.
There are plenty of alternatives for git hosting. The last thing which needs to be migrated before we get an exodus off this vibe coded monstrosity is reputation (i.e. github stars) which are inherently sticky.
Re: Incident with Github.com [resolved]
#424> Update - We are seeing an approximate 20% error rate across numerous experiences including Pull Requests, Issues, and others. Investigations are currently under way and we will be posting updates as they become available brutal
Update: yes that worked. gh CLI using the API was able to make a PR while the web interface was inoperable.
Second update: defeat from the jaws of victory -- the page that displays open PRs is also down.
Third update: CI isn't working anyway, so the CLI PR creation workaround doesn't achieve much.
Re: Incident with Github.com [resolved]
#425We're starting to have the conversation of "do we need to replace GitHub as our CI/CD pipeline?" No idea if/when we decide to pull that bandaid but I think once we can't merge a hot fix into main because of GH being down, it'll be a big org push to replace it. What's everyone else using? Self hosted Gitlab? Gitea?
blacksmith.sh just raised tens of millions on the core premise that self-hosting CI/CD is hard [0]. they've had multiple outages in the last week, so i guess they are proving themselves correct. in practice, giving ssh/sudo to AI (codex) and telling it to setup ephemeral self-hosted runners on your own host VM (qemu), takes about 30 minutes and works extremely well. it bangs out all the shell scripts. moved my own bu…
Re: Incident with Github.com [resolved]
#426Re: Incident with Github.com [resolved]
#427Earlier quoted context omitted.
Huh, they usually do? I'm not a full-time developer though so I'm surely biased towards doing the obvious quick fixes or already-discussed things, but discussing how to solve a problem before throwing something at the wall seems like the normal thing to do for me Or is their point about abandoned projects where the dev never clicks release after merging, or even never merges?
The point is you can't ship shit during an outage.
Re: Incident with Github.com [resolved]
#428Earlier quoted context omitted.
Those are both hilarious to me. In my last FTE role we had the exact opposite beliefs - we wanted to leave GitLabs increases in pricing and move to annual sub, but the reliability of self-hosted runners and the flexibility (and clarity) of the CI system was fantastic.
> Those are both hilarious to me. Why is it hilarious to believe that the industry leader might be better at something than doing it yourself?
Re: Incident with Github.com [resolved]
#429Funny enough, I received a marketing email from GitHub during the outage with the following subject: "What if every pull request you started actually shipped"
Huh, they usually do? I'm not a full-time developer though so I'm surely biased towards doing the obvious quick fixes or already-discussed things, but discussing how to solve a problem before throwing something at the wall seems like the normal thing to do for me Or is their point about abandoned projects where the dev never clicks release after merging, or even never merges?
Re: Incident with Github.com [resolved]
#430I had a lot of goodwill for GitHub but I think today is the tipping point. Looking at a unicorn page, I feel this lingering hope that it's transient (like it usually was in the old days) but my mind reassures me it's probably going to be a long full outage again. The hope is dead.
I’m willing to give them a break as I’m assuming they have a lot of scaling problems due to the influx of LLM assisted coding. But maybe I’m wrong?