Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Monorepoize – Bash scripts for creating a monorepo out of smaller repos

github.com

Post reply on HN
Print view View original

81–90 of 103 posts

Re: Monorepoize – Bash scripts for creating a monorepo out of smaller repos

#81

Post by bdcravens »

koonsolo wrote: post #21
frollo wrote: post #18

After working on a monorepo and then the split up repos for the same codebase, I cannot fathom why somebody would want to take small repos and merge them in a single repo. The mess and complexity just increase.

In git I fully agree, and I wonder which company successfully runs a monorepo in git. For me, I prefer git submodules, which seem to have the benefit of both monorepo and separate repo's.

I've been looking at that approach. We have a large "app" that has a lot of boundaries (parts even being in different languages), but everything is run (love being able to docker-compose up my entire stack) and deployed in unison. However, it does make for a messy single repo.

Re: Monorepoize – Bash scripts for creating a monorepo out of smaller repos

#82

Post by bdcravens »

Kipters wrote: post #28
koonsolo wrote: post #21

Earlier quoted context omitted.

In git I fully agree, and I wonder which company successfully runs a monorepo in git. For me, I prefer git submodules, which seem to have the benefit of both monorepo and separate repo's.

> I wonder which company successfully runs a monorepo in git. Microsoft

Worth noting that due to the size of the Windows repo, they ended up having to extend git significantly

https://devblogs.microsoft.com/bharry/scaling-git-and-some-b...

https://github.com/microsoft/VFSForGit

Re: Monorepoize – Bash scripts for creating a monorepo out of smaller repos

#84

Post by alexbanks »

frollo wrote: post #18

After working on a monorepo and then the split up repos for the same codebase, I cannot fathom why somebody would want to take small repos and merge them in a single repo. The mess and complexity just increase.

I feel like the library needed is actually the opposite. Take a monorepo, do some code analysis, split into many repositories. Monorepos are fool's gold unless you can have a team whose sole job is managing monorepo complexity, and even then I might ask "Why waste a team on monorepos?"

Re: Monorepoize – Bash scripts for creating a monorepo out of smaller repos

#85

Post by ratww »

beagle3 wrote: post #77
ratww wrote: post #70

Earlier quoted context omitted.

> ...until you force them into working in a bad environment where he's treated like... a cog in the machine with zero ability to influence his own productivity, happiness or even the quality of his code.

I disagree (unless your "bad environment" gets tautologically redefined to mean any place where people with good intentions don't get good results.... which you might). I was working in a place that employed incredibly smart and capable but highly opinionated people, and did not treat them as cogs, and did let them influence their productivity, happiness, and quality of the code. However, there was a major philosophi…

If "it worked" and "was not a bad environment", I really don't see the issue here (unless your "it worked" gets redefined to mean "it didn't work"... which you might). Sounds like a nice and intellectually stimulating environment, and that they had best intentions. Honestly, someone wanting a dictator has bigger ego problems than those people.

Re: Monorepoize – Bash scripts for creating a monorepo out of smaller repos

#86

Post by reilly3000 »

juped wrote: post #79
globular-toast wrote: post #76

I'm constantly amazed at how fashions come and go in cycles. Only a few years ago everyone was vigorously splitting everything up into microservices and now the monolithic repository is the latest trend.

Both of those are current trends - microservices in a monorepo - as far as I can tell. (Not a fan of either.)

Right, microservices beget monorepos because managing permissions and build pipelines across 100's of repos isn't pleasant. As mentioned elsewhere in this thread, coordination is really challenging across multiple repos and versions.

Re: Monorepoize – Bash scripts for creating a monorepo out of smaller repos

#87

Post by danenania »

alexbanks wrote: post #84
frollo wrote: post #18

After working on a monorepo and then the split up repos for the same codebase, I cannot fathom why somebody would want to take small repos and merge them in a single repo. The mess and complexity just increase.

I feel like the library needed is actually the opposite. Take a monorepo, do some code analysis, split into many repositories. Monorepos are fool's gold unless you can have a team whose sole job is managing monorepo complexity, and even then I might ask "Why waste a team on monorepos?"

The problem is that we have crammed many related but distinct concepts--versioning, package management, access control, issue tracking, project management, licensing, etc.--into a single envelope called the "repo".

Having a single top level version and commit-log for all an organization's code is a huge win. But that doesn't mean you necessarily want to manage those other more granular concerns at the top level too. With tooling that does a better job drawing these boundaries, we should be able to have the best of both worlds (instead of the worst, which is where we are with the currently dominant repo-management tools).

Re: Monorepoize – Bash scripts for creating a monorepo out of smaller repos

#88

Post by jtompl »

My initial though was: "Oh! This lets me quickly build a repo that merges all the smaller repos using git submodules!"

And then I see, it doesn't do it, it just merges the repos... Why would you do that?

Honestly, I don't know why for years git submodules have had such a bad fame. It works out of the box. You can checkout subrepos but you don't have to if you don't want to. You can set up separate branches, have separate commits, CI/CD workflows for each of the repos.

AND you're still able to have all the monorepo advantages: to lock down the dependencies to an exact version; to let a dev/CI pull all of them at one go.

Why wouldn't you just use git submodules for that?

Re: Monorepoize – Bash scripts for creating a monorepo out of smaller repos

#89

Post by JoshTriplett »

mceachen wrote: post #72
pierrebai wrote: post #63

Earlier quoted context omitted.

git has submodules. A repo can thus host any number of sub-repos. Using modules can also be pretty much like having a monorepo if you have a master repo with all others under it. This way, you can still publish things like public API as a repo.

Submodules don't allow for atomic commits that span repos, though. Atomic cross-project commits are the most compelling selling feature for monorepos IMHO. (A common case for this is changing a function's API, and fixing all downstream consumers, in one tidy, holistic PR).

> Submodules don't allow for atomic commits that span repos, though.

They can, but there's little tooling support to help you do so.

You can make a new commit in one repo, and then in one atomic commit, you can pull that new commit into a parent repo's submodule and fix the users calling into that submodule.

Re: Monorepoize – Bash scripts for creating a monorepo out of smaller repos

#90

Post by Kipters »

vorpalhex wrote: post #69
jedieaston wrote: post #53

Earlier quoted context omitted.

Windows is in one repo at Microsoft, I believe. https://devblogs.microsoft.com/bharry/the-largest-git-repo-o...

Yeah but they basically wrote their own wrapper around it, which is to say Microsoft has fallen into one of their usual patterns: 1. Picking a trendy tool 2. Mis-using trendy tool 3. Rewriting trendy tool so it's no longer trendy tool but something custom and not quite standards-compliant with it's own weird behaviors and bugs 4. Complain the standard is wrong

They wrote a wrapper around it because git took so much time for every basic operations, even a `git status` would take minutes. But still it's not a fork, you can either use it or not without impacting anyone else using the repo. And they also made it available to everyone, as how good open source citizens should do.
Post reply on HN