Ask HN: Why did the number of new GitHub repos crash in late 2022?

#1

Post by Arnehuang » Wed, Oct 18, 2023, 9:54 PM UTC

I was looking at the data from https://ossinsight.io/explore/?id=6c060cc1-83b4-43f7-be4a-a401e4a5387f. I filtered to US only but across the board there was a large crash around August 2022. Curious if anyone knows why.

Re: Ask HN: Why did the number of new GitHub repos crash in late 2022?

#2

Post by fiddlerwoaroof » Wed, Oct 18, 2023, 10:54 PM UTC

People not wanting copilot to be trained on their code?

Re: Ask HN: Why did the number of new GitHub repos crash in late 2022?

#3

Post by altdataseller » Wed, Oct 18, 2023, 11:00 PM UTC

When you see stuff like this happens. It’s most likely something wrong with the tool that is gathering this data. They lost data access or something funky like the max number of pages they can iterate went down

Re: Ask HN: Why did the number of new GitHub repos crash in late 2022?

#4

Post by jjgreen » Wed, Oct 18, 2023, 11:15 PM UTC

Mandatory 2FA announced mid 2022?

Re: Ask HN: Why did the number of new GitHub repos crash in late 2022?

#5

Post by gustavus » Wed, Oct 18, 2023, 11:24 PM UTC

jjgreen wrote: post #4

Mandatory 2FA announced mid 2022?

I'd probably put money on that. Not that it stopped people from creating repos but that it did something with the permissions of the tool gathering the data and the rate limiting to reduce the amount of data ingested.

Re: Ask HN: Why did the number of new GitHub repos crash in late 2022?

#6

Post by mattdeboard » Wed, Oct 18, 2023, 11:29 PM UTC

that query isn’t for new Github repos, it’s for the number of github repos the counter was 1. allowed to count and 2. knew how to count. all that said it is pretty interesting 90k repos went away somehow between August 1 and September 1 last year. Good question. (Also, interesting tool. Never heard of ossinsight before.)

Re: Ask HN: Why did the number of new GitHub repos crash in late 2022?

#7

Post by thunderbong » Thu, Oct 19, 2023, 3:02 AM UTC

Could it be due to [0]:

> Most likely better bot protection, almost all of npm is currently scrapped to teach ai bots coding

[0]: https://old.reddit.com/r/programming/comments/16mna5z/whats_...

Re: Ask HN: Why did the number of new GitHub repos crash in late 2022?

#8

Post by [deleted] » Thu, Oct 19, 2023, 3:57 AM UTC

[deleted]

Re: Ask HN: Why did the number of new GitHub repos crash in late 2022?

#9

Post by SushiHippie » Thu, Oct 19, 2023, 3:59 AM UTC

On clickhouse I can't see that crash. IIRC clickhouse sources their data from https://www.gharchive.org/

The query I used: https://play.clickhouse.com/play?user=play#U0VMRUNUIAogICAgZ...

EDIT: just saw that ossinsight also uses gharchive

I didn't filter for US region though. This is not really published by the github api, only the "location" field is published where users can put everything. ossinsight probably tried to parse the country codes?

But on ossinsight only ~5% of the users have a country code attached to https://ossinsight.io/explore/?id=28da49bd-fec9-4720-a4df-da...