I'm surprised they don't have a couple of separate clusters that they roll things out to and monitor. Seems like you could have a very stable "high paying customers" cluster that is at the very end of your deployment cycle after a ton of canary checks on the way get through.
GitHub degraded performance – resolved
41–50 of 58 posts
Re: GitHub degraded performance – resolved
#42Earlier quoted context omitted.
Yes. It's worth noting that you can take almost any software from before the late 90s to early 2000s, depending on the vendor, that is still available, and with a layer of emulation get it running in minutes. The vast majority of software that is being built today for end users simply will not function in a short time frame because of aggressively built in dependencies on cloud based services, often with those depend…
> I shudder to think how much of the world economy would grind to a halt if Amazon or another significant cloud provider had a sustained, multi-region outage (say 24-48 hours). You can rest at ease. Nothing that is mission-critical for the world's financial infrastructure is hosted within one of these sorts of facilities. Facebook and Netflix might go down for days, but your Amex will still work at any merchant with…
I wish I had your confidence in the financial system, but it's not the financial services I am concerned about.
Financial services are important for moving money around, and processing electronic payments, but it doesn't matter how effectively you can process a wire if there are significant supply chain disruptions, and systems failures that take down the platforms that major retailers and distributors use for logistics.
Even if ATMs and bank networks remain up, what about the encashment and physical security services that those institutions rely on to move around actual physical money?
The economy is more than just financial services, and all of those financial services are just proxies for the real world goods that people need to survive for more than a few days in most urban centres.
Re: GitHub degraded performance – resolved
#43I'm surprised they don't have a couple of separate clusters that they roll things out to and monitor. Seems like you could have a very stable "high paying customers" cluster that is at the very end of your deployment cycle after a ton of canary checks on the way get through.
With Actions they do this, if you're on GitHub Enterprise and run an action it picks machines out of a special pool set aside just for enterprise customers.
Re: GitHub degraded performance – resolved
#44They were down at least a solid hour last week and didn't even post to their status page. I put in a query and got no response. They then unilaterally closed the ticket asked me how my support experience was. Time to move to Gitlab. https://news.ycombinator.com/item?id=28874751
Re: GitHub degraded performance – resolved
#45Earlier quoted context omitted.
> GHE has its own product roadmap that is quite a bit behind the cloud product This is exactly what we want though. We don't need the new fancy shit on a regular cadence. Issues, Code, PRs and 1 line checkbuild scripts are all we care about. Everything else is built into our software.
I'd recommend checking out gitlab for on-prem hosting in an enterprise environment. Works great, integrates with AD/ldap and most of the features are available on the free tier if you want to test it. Practically a drop in replacement for github.
I love GitLab, and am now a shareholder, but advertising it as "drop in replacement" is setting up an evaluator for disappointment
At the very least, they use just absolutely incompatible yaml files for their CI pipelines (in, of course, an incompatible location in the repo)
But probably the bigger obstacle would be their incompatible API (and incompatible auth to it); that means one cannot grab a cool "github bot/tool for doing $X" and expect it to do anything reasonable in GitLab
Re: GitHub degraded performance – resolved
#46Hmm, how do I get the Github Actions CI to run on all the already existing PR's for which it never ran? Anyone know?
Would this do it? https://docs.github.com/en/rest/reference/actions#create-a-w... or, depending on the error, perhaps this? https://docs.github.com/en/rest/reference/actions#re-run-a-w...
Re: GitHub degraded performance – resolved
#47This one is pretty nasty. Getting tired of the disengagement this causes for the team. It's basically a lost day of productivity even if GH goes down for only 30 minutes. Yes, we can continue coding locally, but issues & PRs are a huge part of our daily process. When I get back from vacation we are moving our shit to the enterprise plan. $21/user-month is really not that big of a deal when you are running basically y…
I don't buy this at all. Even if GH is down for the entire day, how the heck does NO-ONE on your team know what they're doing? Was no-one working on something already? Did nobody pay any attention during planning? Do you not have anything you can do from memory from your backlog?!
If you can't work for a day just because Github is down, then there's bigger problems in your process that github being down. I'm sorry of that sounds harsh, but you're either being hyperbolic or you have some real issues to fix in your team or organisation.
Re: GitHub degraded performance – resolved
#48Earlier quoted context omitted.
> so you start finding other ways to occupy your time Ok, pick a ticket and do some work on it locally, when that's done do the same with another. I can go a full day without interacting with github because I'm working on a local branch. Make a note of what branches you need to push later. I can't possibly imagine throwing my arms up and saying the day is wasted because I have to work locally. It's completely unbelie…
I understand your point and agree, but "pick a ticket" can be hard if you use GitHub issues for those.
You could use jira for tickets. Why put all your eggs in one basket?
Re: GitHub degraded performance – resolved
#49This one is pretty nasty. Getting tired of the disengagement this causes for the team. It's basically a lost day of productivity even if GH goes down for only 30 minutes. Yes, we can continue coding locally, but issues & PRs are a huge part of our daily process. When I get back from vacation we are moving our shit to the enterprise plan. $21/user-month is really not that big of a deal when you are running basically y…
> It's basically a lost day of productivity even if GH goes down for only 30 minutes. Are we entering an era where if we don't have hundreds of thousands of servers running 24/7 to host our services, with all the resource consumption and environmental implications that result, that we will no longer be able to remain productive as a society? Is this gradually becoming a new baseline for humanity from which we cannot…
This is like being shocked that so many companies now rely on having an electricity supply and can't work during power outages.