Updated rate limits for unauthenticated requests

What's being strip mined is the openness of the Internet, and AI isn't the one closing up shop. Github was created to collaborate on and share source code. The company in the best position to maximize access to free and open software is now just a dragon guarding other people's coins.

The future is a .txt file of John Carmack pointing out how efficient software used to be, locked behind a repeating WAF captcha, forever.

Several people in the comments seem to be blaming Github for taking this step for no apparent reason.

Those of us who self-host git repos know that this is not true. Over at ardour.org, we've passed the 1M-unique-IP's banned due to AI trawlers sucking our repository 1 commit at a time. It was killing our server before we put fail2ban to work.

I'm not arguing that the specific steps Github have taken are the right ones. They might be, they might not, but they do help to address the problem. Our choice for now has been based on noticing that the trawlers are always fetching commits, so we tweaked things such that the overall http-facing git repo works, but you cannot access commit-based URLs. If you want that, you need to use our github mirror :)

Once again people post in the "community", but nobody official replies; these discussion-pages are just users shouting into the void.

What does “secondary” stand for here in the error message?

> You have exceeded a secondary rate limit.

Edit and self-answer:

> In addition to primary rate limits, GitHub enforces secondary rate limits

(…)

> These secondary rate limits are subject to change without notice. You may also encounter a secondary rate limit for undisclosed reasons.

https://docs.github.com/en/rest/using-the-rest-api/rate-limi...

Wow, I'm realizing this applies to even browsing files in the web UI without being logged in, and the limits are quite low?

This rather significantly changes the place of github hosted code in the ecosystem.

I understand it is probably a response to the ill-behaved decentralized bot-nets doing mass scraping with cloaked user-agents (that everyone assumes is AI-related, but I think it's all just speculation and it's quite mysterious) -- which is affecting most of us.

The mystery bot net(s) are kind of destroying the open web, by the counter-measures being chosen.