One thing I really dislike about monorepos for node modules is that you can't npm install from them directly. Unless it's a project with very fast PR merging and releases you can be stuck with a broken module that has an open or even merged PR to fix it that you can't install because it hasn't been pushed to npm. npm link might work locally, but that doesn't help if it needs to build on a CI server. If it's one repo per module then I can just npm install the git url and it works fine.
Monorepos: Please don’t
101–110 of 402 posts
Re: Monorepos: Please don’t
#102At least the author gave us the courtesy of italicizing his broken assumption from the outset of the post. > Because, at scale, a monorepo must solve every problem that a polyrepo must solve, with the downside of encouraging tight coupling, and the additional herculean effort of tackling VCS scalability. Right. But you have to get to "scale" first (as it relates to VCSs). Most companies don't. Even if they're success…
You can't split monorepos after the fact, at least not without immense costs. You can always just put all your small repos into a big one.
> You can always just put all your small repos into a big one.
It's not quite as simple as that. You'll need to avoid rebuilding the entire repo for every change - using something like Bazel. This means your build tooling has to be replaced entirely, which is a non-trivial task and not something your devops/release engineering team will thank you for.
For any 3rd party libraries used by your projects you need to either ingest those projects into your monorepo and update forever. Or keep npm/maven/pip/gem/whatever around just for managing 3rd party dependencies (+ whatever system you use to front the main language package registries, because of course you're not talking directly to NPM/Maven central are you? What if they go down or do a leftpad?).
I think either system - monorepos or polyrepos - works fine; just pick one and stick with it. Monorepos will probably give you better velocity starting out. Past a certain size, which most software shops will never hit, the already-available tools lend themselves better to polyrepo. And more devs know polyrepo tools (eg. Jenkins) than the corresponding monorepo solution (eg. Bazel). Things might swing in favor of monorepo on the VCS side if Twitter/Google/FB ever open-source their stuff.
Re: Monorepos: Please don’t
#103Earlier quoted context omitted.
Can a monorepo support module- or subdirectory-level ownership controls? Or do teams using a monorepo just do without them? Partially answering my own question: SVN, recommended in a prior comment [0], supports path-based authorization [1]. But what about teams using another version control system? [0] https://news.ycombinator.com/item?id=18810313 [1] http://svnbook.red-bean.com/en/1.5/svn-book.html#svn.serverc...
If I remember correctly, Gitlab has introduced some sort of ownership control where you can say who owns what directories for things like approving merge requests that affect those directories.
Github has this feature[1], which we use extensively in our monorepo.
Re: Monorepos: Please don’t
#104My last 2 jobs have been working on developer productivity for 100+ developer organizations. One is a monorepo, one is not. Neither really seems to result in less work, or a better experience. But I've found that your choice just dictates what type of problems you have to solve. Monorepos are going to be mostly challenges around scaling the org in a single repo. Polyrepos are going to be mostly challenges with coordi…
Sounds reasonable. I'll have to add, though, that the underlying technology factors into this as well.
For example: If you're stuck with a TFS monorepo (you poor soul), you actually get to deal with both problems to some extent, since TFS doesn't enforce that you check out the intire repository at once.
This can have very "funny" situations because someone forgot to checkout new changes in some folder. OTOH, at least for releases, you can remedy this by using CI everywhere.
Re: Monorepos: Please don’t
#105Overly opinionated garbage, imagine having to work with this guy.
Re: Monorepos: Please don’t
#106My advice is that if components need to release together, then they ought to be in the same repo. I'd probably go further and say that if you just think components might need to release together then they should go in the same repo, because you can in fact pretty easily manage projects with different release schedules from the same repo if you really need to. On the other hand if you've got a whole bunch of component…
After trying to get the best of both with Subversion Externals and Git Submodules, I'd have to agree. At least until things are so loosely coupled they're begging for a public release. That said, some packaging solutions can bridge the gap reasonably well. Unless you need instantaneous, atomic releases.
I switched to using submodules about a year ago, and they work very well for a project + a set of 4 dependencies. I handle that zoo from VS Code + Git Lens plugin.
Funnily, I only use Code to handle commits to submodules, because Git Lens is not available for the full VS IDE.
Re: Monorepos: Please don’t
#107Earlier quoted context omitted.
You can't split monorepos after the fact, at least not without immense costs. You can always just put all your small repos into a big one.
I think there's a nuance to this that should be pointed out: Monorepos allow you to do very bad hacks (I need this other component over there; let me just put in a Symlink. Done.). And if people can, they will use those hacks. If you split your repo up from the get go, the worst thing you can get that you'll have to assemble multiple distinct, well-encapsulated (in terms of project structure) things into one. In Git,…
> Monorepos allow you to do very bad hacks (I need this other component over there; let me just put in a Symlink. Done.).
Why would you put in a symlink? You could just provide a path to the actual component and import it into your project.
> the worst thing you can get that you'll have to assemble multiple distinct, well-encapsulated (in terms of project structure) things into one
When you have multiple repos, you also have multiple versions and releases of things. Now every team has the following options:
1. Backport critical fixes to every version still in use (hard to scale)
2. Publish a deprecation policy, aggressively deprecate older releases, and ignore pleading and begging from any team that fails to upgrade (infeasible - there'll always be a team that can't upgrade at that moment for some reason)
You also have to solve the conflicting diamond dependency problem. This is when libfoo and libbar both depend on different versions of libbarfoodep. It's even more fun in Java because everything compiles and builds, but fails at runtime. So now you have to add rules and checks to your entire dependency tree - some package managers have this (Maven), others don't (npm IIRC).
Re: Monorepos: Please don’t
#108Just read the article and it was really great read. We're using polyrepos and dealing with so many repos was not great. That's why I created "gitbatch". Gitbatch allows you to manage multiple repositories in an easy way.
Re: Monorepos: Please don’t
#109My advice is that if components need to release together, then they ought to be in the same repo. I'd probably go further and say that if you just think components might need to release together then they should go in the same repo, because you can in fact pretty easily manage projects with different release schedules from the same repo if you really need to. On the other hand if you've got a whole bunch of component…
Components might need to be released “together”, but if they are worked on by different teams, it means they’ll have a different release process, as in different timeline, different priorities.
First of all this is normal, because otherwise the development doesn’t scale.
In such a case the monorepo starts to suck. And that’s the problem with your philosophy ... it matters less how the components connect, it matters more who is working on it.
Truth of the matter is that the monorepo encourages shortcuts. You’d think that the monorepo saves you from incompatibilities, but it does so at the expense of tight coupling.
In my experience people miss the forest from the trees here. If breaking compatibility between components is the problem, one obvious solution is to no longer break compatibility.
And another issue is one of responsibility. Having different teams working on different components in different repos will lead to an interesting effect ... nobody wants to own more than they have to, so teams will defend their components against unneeded complexity.
And no, you cannot split a monorepo into a polyrepo easily. Been there, done that. The reason is that working in a monorepo versus multiple repos influences the architecture quite a lot and the monorepo leads to very unclear boundaries.
Re: Monorepos: Please don’t
#110Earlier quoted context omitted.
It's an interesting social problem in how you manage those project / library / repository boundaries. On the flipside, though, it's been well documented that among many of the major monorepos those boundaries still exist, they just become far more opaque because no one has to track them. You find the weird gatekeepers in the dark that spring out only when you get late in your code review process because you touched "…
Can a monorepo support module- or subdirectory-level ownership controls? Or do teams using a monorepo just do without them? Partially answering my own question: SVN, recommended in a prior comment [0], supports path-based authorization [1]. But what about teams using another version control system? [0] https://news.ycombinator.com/item?id=18810313 [1] http://svnbook.red-bean.com/en/1.5/svn-book.html#svn.serverc...
google3 uses the same model that Chromium does, see an example here: https://github.com/chromium/chromium/blob/master/chromeos/OW...