I would maybe start migrating repos away from GitHub, since they have proven to be quite unreliable. Nonetheless, I must say I do appreciate GitHub's UI and even their CLI is quite nice. Is there a service out there that basically provides the same good user experience, CI/CD and has no costs for public repos?
Apart from the unreliability I can't complain about GitHub.
Incident with GitHub Actions
11–20 of 50 posts
Re: Incident with GitHub Actions
#12I think it's easy to pile-on and say "GitHub is down again! Should've self hosted lol!".
When, in reality, it's one service having issues and not the whole site. These incidents also seem to be resolved quickly.
Downtime is not the end of the world.
Re: Incident with GitHub Actions
#13I'm glad we have a self-hosted Gitlab. Sure you need to do a bit of setup and configuration, but it's worth in the long run.
Doesn't it cause more headaches? I've had experiences where devops minded people I've worked with have wanted to self-host services such as Bitwarden. Sure, it will be cheaper and you will have noone to blame but yourself, but once things go bad they go really bad. It's also another thing to keep eyes on. I guess similar argument could also be extended to self-hosted clouds. Seems like it could take away a lot of foc…
> Doesn't it cause more headaches?
No. You update it on patch day (or when a big CVE comes out that you are actually impacted by) and know exactly what goes wrong and when. If you can't solve it, you roll back. When Github (or a part of it) goes down, you know nothing and with persistent issues there's no way to solve it either.
A company of the size where "but we have to scale" is an actual issue should self-host. A SaaS solution is a risk that you cannot mitigate.
Re: Incident with GitHub Actions
#14Oh it's just the daily GitHub incident.
Re: Incident with GitHub Actions
#15I'm glad we have a self-hosted Gitlab. Sure you need to do a bit of setup and configuration, but it's worth in the long run.
It's a bit of a weird thing as the whole word tries (or tried) to move away from on-prem stuff. Like Jira stopping support for non-cloud versions etc.
Re: Incident with GitHub Actions
#16As the person who gets pinged when our CI isn't working, I'm hoping my next employer doesn't have github in their stack.
We've reached the stage where architectural decisions are being questioned. Nobody should need to ask me "well why are we using github?" but here we are.
They needed to fix this two weeks ago. The next time I'm in charge of stack decisions I will be evaluating competitors. This is exactly how Slack beat Hipchat.
Don't let weekends dilute your view of the situation. There's only 23 weekdays in March and Github has not been reliable for 5 of them.
Re: Incident with GitHub Actions
#17I think it's easy to pile-on and say "GitHub is down again! Should've self hosted lol!". When, in reality, it's one service having issues and not the whole site. These incidents also seem to be resolved quickly. Downtime is not the end of the world.
no but in a team of thousands of developers it costs a hell of a lot of money in lost productivity.
Re: Incident with GitHub Actions
#18At this point in time it really is a gamble if Actions work or not. Even if there is no “Incident” the success rate is probably at 70-80% in the last three weeks.
Re: Incident with GitHub Actions
#19I'm glad we have a self-hosted Gitlab. Sure you need to do a bit of setup and configuration, but it's worth in the long run.
Doesn't it cause more headaches? I've had experiences where devops minded people I've worked with have wanted to self-host services such as Bitwarden. Sure, it will be cheaper and you will have noone to blame but yourself, but once things go bad they go really bad. It's also another thing to keep eyes on. I guess similar argument could also be extended to self-hosted clouds. Seems like it could take away a lot of foc…
A lot of large web services outages (such as GitHub, Azure Active Directory, Slack, etc) are purely caused by these services having to scale to the entire world, with all the complexity and moving parts it entails.
Self-hosting inherently mitigates that problem because you now need to support less than 0.1% of the load of the worldwide service.
It also puts you in control of maintenance and updates - you can choose to make changes outside of business hours so that nobody is affected if you screw up. Developers at SaaS services can't easily do that because it's always business hours in some parts of the world, and may not be motivated to do it anyway even if it was possible with some effort.
Re: Incident with GitHub Actions
#20I think it's easy to pile-on and say "GitHub is down again! Should've self hosted lol!". When, in reality, it's one service having issues and not the whole site. These incidents also seem to be resolved quickly. Downtime is not the end of the world.
Until you your self have an issue you cannot fix because github is 'having issues' so you cannot build & deploy. I'm not saying self-host everything, even though I prefer it for many things, but do make sure you have redudancy in these processes. When Github has issues, out company stops. That is not an acceptable dependency imo
> Downtime is not the end of the world.
Tell that to our customers & support...