Ask HN: How to handle 500+ repositories in GitHub?

#1

Post by voorloopnul » Mon, Jul 12, 2021, 11:13 AM UTC

Searching for ways to handle a large number of repositories I stumbled in this thread: https://github.community/t/structuring-repositories-or-organisations/817

Apparently Gitlab and Bitbucket give you a feature to group repositories into projects while Github is lacking something similar.

How companies using Github handle their repositories when there are 100's of them?

Re: Ask HN: How to handle 500+ repositories in GitHub?

#2

Post by rurban » Mon, Jul 12, 2021, 11:47 AM UTC

I have currently 375 repos under my user, plus about 20 more under different orgs. What's the exact problem to solve? Just avoiding to pay per user per org?

I even have some crontab's to update all issues via git bug bridge locally, so that I don't need an internet connection to github.com to work on tickets. Clicks are not easily automated, so I avoid them for my workflows. Same for almost daily rebasing. (all feature and bugfix branches are automatically rebased to master, when master moves). This is 90% automatic.

All my repos are public, I pay nothing. I find that better than paying for it and keeping them private.

Re: Ask HN: How to handle 500+ repositories in GitHub?

#3

Post by PaywallBuster » Tue, Jul 13, 2021, 5:51 AM UTC

rurban wrote: post #2

I have currently 375 repos under my user, plus about 20 more under different orgs. What's the exact problem to solve? Just avoiding to pay per user per org? I even have some crontab's to update all issues via git bug bridge locally, so that I don't need an internet connection to github.com to work on tickets. Clicks are not easily automated, so I avoid them for my workflows. Same for almost daily rebasing. (all featu…

"feature to group repositories into projects while Github is lacking"

Re: Ask HN: How to handle 500+ repositories in GitHub?

#4

Post by captn3m0 » Tue, Jul 13, 2021, 5:59 AM UTC

For managing them, we have a wrapper over terraform-github-provider[0]. There's also ghconf[1].

Unfortunately, no solutions for grouping them.

[0]: https://registry.terraform.io/providers/integrations/github/...

[1]: https://github.com/optile/ghconf

Re: Ask HN: How to handle 500+ repositories in GitHub?

#5

Post by noufalibrahim » Tue, Jul 13, 2021, 6:06 AM UTC

Not exactly your requirement but I heavily use Github Classroom while teaching and it generates a ton of repositories (one per exercise per student). I name them consistently and wrote a few command line scripts to grade, delete etc. them.

Re: Ask HN: How to handle 500+ repositories in GitHub?

#6

Post by cik » Tue, Jul 13, 2021, 6:06 AM UTC

Unfortunately we ended up giving in and using multiple GitHub organizations. There doesn't seem to be a better answer like repository tagging or labeling. It's not ideal, and has definitely resulted in lost issues, duplicated effort and the like. However - it's the best answer we have.

Re: Ask HN: How to handle 500+ repositories in GitHub?

#7

Post by sverhagen » Tue, Jul 13, 2021, 6:06 AM UTC

We are currently moving from Bitbucket to GitHub (hurrah!), and since we were "assigned" a single organization in GitHub by our corporate parent, we are also looking at importing a lot of repositories into a single organization. And the inability to further organize these repositories has been a big disappointment for an otherwise good experience with GitHub. Pretty much what this thread is about!

We end up telling our teams to use the different search functions to help in making sense of the madness: search by language (automatically detected by GitHub), team, type (forks or other), topics and names. We have a decent process to make sure the appropriate topics are set for repositories (we manage all non-forks through Terraform), and to make sure that names include a useful prefix. Though I don't think we'll ever stop debating the right prefixes for our repository names.

It's a "poor man's" ordeal, that's for sure. Organizations are probably the right way to go, if you can.

We're still in the move, so too soon to tell, but I am also wondering if we aren't worried about a non-issue. Because while Bitbucket has "groups", I don't think I have ever consciously used those groups while working in Bitbucket (nor have we ever set up the Bitbucket groups very well for our organization anyway).

Re: Ask HN: How to handle 500+ repositories in GitHub?

#8

Post by bramblerose » Tue, Jul 13, 2021, 6:07 AM UTC

In the process of moving from GitLab to GitHub. We just use naming conventions for this: all repositories are called Company.Group.Product, where Group roughly corresponds to the old GL group.

Re: Ask HN: How to handle 500+ repositories in GitHub?

#9

Post by andrewstuart2 » Tue, Jul 13, 2021, 6:13 AM UTC

bramblerose wrote: post #8

In the process of moving from GitLab to GitHub. We just use naming conventions for this: all repositories are called Company.Group.Product, where Group roughly corresponds to the old GL group.

Just out of curiosity, why the move? I'll admit I'm something of a GitLab fanboy, but honestly it's because they keep winning on just about everything in CE vs even the paid alternatives (in my mind).

Re: Ask HN: How to handle 500+ repositories in GitHub?

#10

Post by elboulangero » Tue, Jul 13, 2021, 6:16 AM UTC

I use this command-line tool: https://myrepos.branchable.com/