241–250 of 268 posts
Earlier quoted context omitted.
In my experience microservices are easier to manage and understand when organized in a monorepo.
That indicates a strong coupling between those microservices.
I've worked for a company with a large monorepo. At first I was a fan, but now I'm not so sure. The web of dependencies was too much. Now I think teams should be allowed to reuse other teams' code only as libraries or APIs with actual release cycles. There shouldn't be any "oh let's depend on the HEAD of this random build target somewhere else in the monorepo". There should be only "let's depend on a released version…
This added friction means you will do something unsavory to rush out a fix for yourself instead of fixing it in the dependency, waiting for a release, bumping the version of your dependency, then finally making your own release.
Earlier quoted context omitted.
> there a way to set permissions on certain directories / force partial clones. No. And that's one reason small startups should separate frontend code into a separate monorepo. If you would like to hire a contractor for SEO/web developer then give them access to frontend code. Keep the backend code segmented out.
This is exactly my point. I like git, I like monorepos, but I do care about control over access and history. I use git mainly because everybody knows it, tooling is there, etc.
This thread is reminding me of a prior one about complexity merchants. I am seeing a lot of sentiment that there is somehow a technical sacrifice by moving to a monorepo. This is absolutely ludicrous unless you fail to grasp the power of a hierarchical file system. I don't see how a big mess like CI/CD is made easier by spreading it out to more points of configuration. To me the whole point of a monorepo is atomic co…
While things like git submodules have problems, are they not sort of the best of both worlds camp? I never quite got the arguments/grumblings about this stuff but perhaps I do not understand what friction people are hitting up agains. It feels to me the problems of multiple repositories is the same problem of having to manage versions of dependencies like in pip or something that you do not even own. Perhaps people a…
One top level "superrepo" is the best of both worlds. You don't break the limits of git, you can include third party repos, you can seal off access to some secret repos. All while still giving benefit of everything checked out in one place. Tooling for flattening the git log exists and is relatively cheap to build yourself, It's easy to design CI that runs both big and small jobs in the superrepo and the subcomponents.
Nested submodules = Game over. Don't ever do it. This is the worst of both worlds. Common components included twice or more need to be aligned to exact same revision, without help of semantic versioning. Components far down the dependency tree need to be uplifted recursively through a chain of submodules, resulting in 10 commits with no common CI for one tiny change.
Earlier quoted context omitted.
Great call out. Amazon has an extremely effectively polyrepo setup and it’s a shame there’s no open source analog. Probably because it requires infrastructure outside of the repo software itself. I’ve been toying around with building it myself but it’s a massive project and I don’t have much free time.
The Amazon poly-repo setup is an engineering marvel, and a usability nightmare, and doesn't even solve all the major documented problems of poly-repos. The "version set" idea was probably revolutionary when it was invented, but everyone I know who has ever worked at amazon has casually mentioned that their team has at least one college-hire working 25%+ time on keeping their dependency tree building.
Earlier quoted context omitted.
I think one reason is that there are various big companies (Google, Microsoft, Meta) who have talked about the tech they've deployed to make monorepos work, but I at least have never seen an equivalent big successful company describe their polyrepo setup, how they solved the pain points and what the tech around it looks like.
I keep meaning to write a blog post... The short answer, start with a package management system like conan or npm (we rolled our own - releasing 1.0 the same month I first heard of conan which was then around version 0.6 - don't follow our example). Then you just need processes to ensure that everyone constantly has the latest version of all the repos they depend on - which ends up being a full time job for someone t…
One full time job equivalent can buy a lot of tooling. Tooling that not only replaces this role but also shifts the feedback a lot closer to dev introducing the breaking change.
Here's what I never got about monorepos: Imagine you have an internal library and also two consumers of that library in the repo. But then you make breaking changes to the library but you only have time to update one of the consumers. Now how can the other consumer still use the old version of that library?
There's a second option, not mentioned by the sibling comments so far: Publish the library somewhere (e.g. an internal Nexus) and then have the other consumer pin the old version of the library instead of referring to the newest version inside the monorepo. Whether or not this is acceptable is largely a question of ownership.
Earlier quoted context omitted.
You don't make breaking changes. You provide the new API and the old API at the same time, and absorb the additional complexity as the library owner. Best case scenario everyone migrates to the new API and eventually remove the old one. This sounds onerous, but keep in mind at a certain scale there is no one commit in production at any given time. You could never roll out an atomic breaking change anyway, so going th…
Thank you for the response! Genuine question: if you can't have one commit in production at any given time, what advantages for the monorepo remain?
That might be possible in a simple library + 1 consumer scenario if you follow the other commentors' recommendation to always update library + consumer at once. But in many cases you can't, anyway, because you're deploying several artifacts or services from your monorepo, not just one. So while "1 commit in production at any given time" is certainly neat, it wouldn't strike me as the primary goal of a monorepo. See also this discussion about atomicity of changes further up: https://news.ycombinator.com/item?id=44119585
> what advantages for the monorepo remain?
Many, in my opinion. Discoverability, being able to track & version all inter-project dependencies in git, homogeneity of tooling, …
See also my other comment further up on common pains associated with polyrepos, pains that you typically don't experience in monorepos: https://news.ycombinator.com/item?id=44121696
Of course, nothing's free. Monorepos have their costs, too. But as I mention in the above comment and also in https://news.ycombinator.com/item?id=44121851, a polyrepo just distributes those costs and makes them less visible.
Earlier quoted context omitted.
There's a second option, not mentioned by the sibling comments so far: Publish the library somewhere (e.g. an internal Nexus) and then have the other consumer pin the old version of the library instead of referring to the newest version inside the monorepo. Whether or not this is acceptable is largely a question of ownership.
Thanks for your response! Don't you lose the main advantage of the monorepo then, since you can no longer rely on the fact that all the code at any one commit in history fits together? Or are there other significant advantages that still remain?
The point about trying to stick with a single language build tooling really cannot be stressed enough. It is what prompted me to write a simplified version of Bazel, a generic "target determinator" with caching capabilities if you will. I call it "Grog", the monorepo build tool for the grug-brained developer. https://grog.build/why-grog/
If a single language is an option you are a small project that is not facing the problems people on large projects are facing. A monorepo will be easy for you without read the article and the lessons learned. Come back when you have millions of lines of code, written over decades by hundreds (or thousands) of full time developers.