Incident with Issues and Pull Requests • Hacker News Board

Re: Incident with Issues and Pull Requests

#51

Post by candiddevmike » Thu, May 11, 2023, 1:50 PM UTC

dbingham wrote: post #20

What is going on over there? Third day in a row is... kind of impressive.

Today seems worse than yesterday. I'm getting wildly inconsistent results when viewing repositories after a push. Hard to tell if my push actually went through, and it's not triggering actions.

Re: Incident with Issues and Pull Requests

#52

Post by SideburnsOfDoom » Thu, May 11, 2023, 1:50 PM UTC

dbingham wrote: post #20

What is going on over there? Third day in a row is... kind of impressive.

It comes in around 09:30 on US east coast.

I suspect that it's related to high load.

Re: Incident with Issues and Pull Requests

#53

Post by galleywest200 » Thu, May 11, 2023, 1:51 PM UTC

buglungtung wrote: post #43

I'm considering host a gitea instance backup all of my repos. I have an important fix that need to be deployed right now but there is no way to deploy it in a normal way with our CI which one was setup with Github Action. Fortunately I have a instruction to bypass CI and build the source by myself. But again, Github defeat me because our release workflows are depend on GitOps which are effected by Github issue. Ahhhh…

Gitea's ability to create a local repository as mirror of a remote repository is great for this. You can stay on Github and have your code regularly mirrored locally.

Re: Incident with Issues and Pull Requests

#54

Post by robofanatic » Thu, May 11, 2023, 1:51 PM UTC

candiddevmike wrote: post #51

dbingham wrote: post #20

What is going on over there? Third day in a row is... kind of impressive.

Today seems worse than yesterday. I'm getting wildly inconsistent results when viewing repositories after a push. Hard to tell if my push actually went through, and it's not triggering actions.

exact same issue

Re: Incident with Issues and Pull Requests

#55

Post by samwillis » Thu, May 11, 2023, 1:51 PM UTC

Hugs to the GitHub Opps and SRE teams right now!

Also hugs to any Devs, Opps or SREs directly effected by this outside GitHub.

Looking forward to a post-mortem on the last few days, I'm sure it will be a really interesting read.

Re: Incident with Issues and Pull Requests

#56

Post by capableweb » Thu, May 11, 2023, 1:51 PM UTC

atl4s wrote: post #17

Just lost a merge commit to dev/null. This is getting tiresome

How??? You do the merge, which either creates a new commit for the change, or appends the commits to your existing tree. Then you push that to the remote. If the push fails, you can just push again, it's not lost. And if the merge failed, you didn't have any merge commit to begin with.

Re: Incident with Issues and Pull Requests

#57

Post by melx » Thu, May 11, 2023, 1:51 PM UTC

Shout out to all devs that put the deps of their programs on GH...

Right now I can ignore that my PRs show 500 error, or old code best case scenario.

But... I cannot build and ship due to project's dependency depending on some stuff hosted on GH.

Re: Incident with Issues and Pull Requests

#58

Post by mminer237 » Thu, May 11, 2023, 1:52 PM UTC

git says my code pushed fine and everything is synced, but Github is not showing any of my changes.

Edit: 10 minutes later, the Github finally shows the push, but triggers still aren't working.

Edit #2: Things are working normally now.

Re: Incident with Issues and Pull Requests

#59

Post by capableweb » Thu, May 11, 2023, 1:52 PM UTC

Vermyndax wrote: post #13

nickthesick wrote: post #6

Have they said what has been up lately?

They are far overdue for issuing a statement.

They're having so many outages that they've moved from making statements after each event to aggregating them all and write about all of them on a month-by-month basis.

I'm not joking: https://github.blog/tag/github-availability-report/

Re: Incident with Issues and Pull Requests

#60

Post by jgadelange » Thu, May 11, 2023, 1:53 PM UTC

cloudking wrote: post #32

Anecdotal - I've been using Gitlab for a few years on some projects, and haven't experienced any downtime issues with them of this magnitude.

I can still remember https://about.gitlab.com/blog/2017/02/10/postmortem-of-datab... (though I can't rembember reading of any big problems since then)