Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Incident with Issues and Webhooks – Resolved

githubstatus.com

Post reply on HN
Print view View original

101–110 of 275 posts

Re: Incident with Issues and Webhooks – Resolved

#101

Post by skylerwiernik »

eddyg wrote: post #77
AlexB138 wrote: post #15

Github has published some incredible usage rate increase numbers, which they ascribe to the rise of agentic coding. At some point, they are going to have to change rate limits, cut free-tier usage, or find some other path to reducing load. It's clear that their infrastructure can't keep up with this significant increase, and it's unlikely that they're going to just absorb the increased costs themselves. Very curious…

From the GitHub COO on April 3rd: Platform activity is surging. There were 1 billion commits in 2025. Now, it's 275 million per week, on pace for 14 billion this year if growth remains linear (spoiler: it won't.) GitHub Actions has grown from 500M minutes/week in 2023 to 1B minutes/week in 2025, and now 2.1B minutes so far this week. So we're pushing incredibly hard on more CPUs, scaling services, and strengthening G…

This is extremely interesting how fast this happened. Either AI use surged massively in the last quarter, or this is a very sneaky move by Anthropic. Looking at my own stats, I don't think I'm using Claude Code much more than I used to, but my commits have gone way up. I have a feeling they've tuned the models recently to commit more often, which gives the illusion of more work being done.

Re: Incident with Issues and Webhooks – Resolved

#102

Post by tardedmeme »

pluc wrote: post #83
gejose wrote: post #30

Github has 84.92% uptime in the last 90 days according to https://mrshu.github.io/github-statuses I don't know how this is even remotely close to acceptable.

It isn't. Lots of unacceptable things going on these days and everyone seems to be accepting them just fine.

We should make an alternative git site, but how to acquire users?

Re: Incident with Issues and Webhooks – Resolved

#103

Post by incognito124 »

mohsen1 wrote: post #82
AlexB138 wrote: post #15

Github has published some incredible usage rate increase numbers, which they ascribe to the rise of agentic coding. At some point, they are going to have to change rate limits, cut free-tier usage, or find some other path to reducing load. It's clear that their infrastructure can't keep up with this significant increase, and it's unlikely that they're going to just absorb the increased costs themselves. Very curious…

The same company operates the Xbox network. More daily active users and more events per second

Xbox network was _designed_ for such concurrency, GitHub is Ruby on rails + vitess (mysql).

Re: Incident with Issues and Webhooks – Resolved

#104

Post by wavemode »

amluto wrote: post #52
AlexB138 wrote: post #15

Github has published some incredible usage rate increase numbers, which they ascribe to the rise of agentic coding. At some point, they are going to have to change rate limits, cut free-tier usage, or find some other path to reducing load. It's clear that their infrastructure can't keep up with this significant increase, and it's unlikely that they're going to just absorb the increased costs themselves. Very curious…

For literally decades, I’ve observed that there are systems that make each operation cheap and systems that work hard to scale out. The former frequently seems to wildly outperform the latter. GitHub, for example, seems to implement the main repository /pulls page as a search query, which is hinted at by the prefilled search bar and was mostly confirmed last week when the search backend failed and pull requests didn’…

Git itself is kind of a fundamentally computationally inefficient way to store and retrieve information. If the problem to solve were simply "store and version this text", 14 billion commits in a year would not even be considered a lot.

In other words, a centralized version control system built from the ground up to operate at scale would do far more for scalability than anything GitHub could possibly do to optimize their Git operations. Every major tech company (Amazon, Meta, Google, etc) is already doing something like this internally.

Though this would require people to start using a github-specific client rather than the traditional git+ssh. (Though the github client could still maintain a git repo locally, for compat.)

Re: Incident with Issues and Webhooks – Resolved

#105

Post by tardedmeme »

petcat wrote: post #21
indianhippie wrote: post #8

This is reaching an unacceptable level of performance. There isn't a week that work isn't interrupted by GH.

AI agents have changed the scalability properties of basically the entire internet. It used to be that GitHub could rely on a finite number of people interacting with their platform in real human ways in real observable patterns. So I'm assuming that they scale for those patterns, and optimize for the UI and UX hotspots. But now everyone's got a moltbot running 24/7, sometimes many, and it's completely overloading a…

It's not new, it's just a DoS, which is a serious crime, just report the attacker to the police if in your country, or block their IP if not. If done accidentally, it's likely not a crime but the police will still scare them to stop doing it.

Re: Incident with Issues and Webhooks – Resolved

#106

Post by pulkas »

they are testing the limits of ruby on rails.

"Ultimately, if more companies treated the framework as an extension of the application, it would result in higher resilience and stability. Investment in Rails ensures your foundation will not crumble under the weight of your application. Treating it as an unimportant part of your application is a mistake and many, many leaders make this mistake." https://github.blog/engineering/architecture-optimization/bu...

Re: Incident with Issues and Webhooks – Resolved

#107

Post by bravetraveler »

wolfi1 wrote: post #95
eddyg wrote: post #77

Earlier quoted context omitted.

From the GitHub COO on April 3rd: Platform activity is surging. There were 1 billion commits in 2025. Now, it's 275 million per week, on pace for 14 billion this year if growth remains linear (spoiler: it won't.) GitHub Actions has grown from 500M minutes/week in 2023 to 1B minutes/week in 2025, and now 2.1B minutes so far this week. So we're pushing incredibly hard on more CPUs, scaling services, and strengthening G…

I wonder how many of those actions are really necessary

Or how many pushes those commits are spread across; oh, neat, big number.

Re: Incident with Issues and Webhooks – Resolved

#108

Post by sofixa »

pydry wrote: post #39
AlexB138 wrote: post #15

Github has published some incredible usage rate increase numbers, which they ascribe to the rise of agentic coding. At some point, they are going to have to change rate limits, cut free-tier usage, or find some other path to reducing load. It's clear that their infrastructure can't keep up with this significant increase, and it's unlikely that they're going to just absorb the increased costs themselves. Very curious…

Github naturally scales horizontally. Usage numbers is the PR reason. Vibecoding insanity in Microsoft is the more plausible actual culprit.

> Github naturally scales horizontally

Not necessarily, a few years ago they had some crucial information stuck in a single MySQL cluster (so write constrained) and were working on sharding it, but struggling: https://github.blog/engineering/infrastructure/partitioning-...

Re: Incident with Issues and Webhooks – Resolved

#110

Post by steve1977 »

mohsen1 wrote: post #82
AlexB138 wrote: post #15

Github has published some incredible usage rate increase numbers, which they ascribe to the rise of agentic coding. At some point, they are going to have to change rate limits, cut free-tier usage, or find some other path to reducing load. It's clear that their infrastructure can't keep up with this significant increase, and it's unlikely that they're going to just absorb the increased costs themselves. Very curious…

The same company operates the Xbox network. More daily active users and more events per second

Do they run Xbox network on Azure or is it a separate thing?
Post reply on HN