I feel like I’m missing something. Never used monorepo before but looking into it for our node code base. It seems like the combination of a monorepo and individually publishing the packages to a private npm server such as nexus eliminates many of these issues. One project can remain using an older version while another can use a breaking change. Pin your versions and follow semver and you should be fine. What am I missing here?
Monorepo is great if you're really good
131–140 of 159 posts
Re: Monorepo is great if you're really good
#132What author misses is that multi-repo is also not dumbass proof. Our team of about 60 people owned about 5-6 git repos that compiled into one SO. Every simple bug fix required multiple diffs to multiple repos.
The real argument that the author is making is that worse is better. Monorepo requires good tooling and a disciplined team. Multi-repo is worse but it is easier to manage when dealing with inexperienced programmers who want to have their own repo for their shiny little microservice.
IMO, the difference is that multi-repo has limitations and mono-repo has challenges. You will never get atomic commits and precise versioning in multi-repo. With mono-repo, there are a lot of challenges that can be solved with good engineering.
Re: Monorepo is great if you're really good
#133Let's assume without proof that Glibc developers are good. Similarly, kernel.org developers are good as are the gcc.gnu.org people. Should glibc, gcc and the kernel be in a monorepo? (Cue laugh track ...)
That is actually one of the advantages of the BSDs. Kernel and base system are in one repo and released together.
It simplifies development and testing. and an easier upgrade for end-users.
Re: Monorepo is great if you're really good
#134Earlier quoted context omitted.
> There are few organizations that have so much code they break available source control solutions but simultaneously don't have the technical expertise to manage a monorepo that large. Facebook's Mononoke ( https://github.com/facebookexperimental/mononoke ) pretty much removes that argument. They outgrew their current source control, and that's their path forward for the next couple orders of magnitude.
Not sure what you're implying about mononoke but the readme says: > The version that we provide on GitHub does not build yet. So, maybe eventually. They found that Git didn't do well at their scale, so they modified mercurial instead. I'm all in favor of a company-wide mono-repo if it doesn't have scaling issues.
Most of monorepo tools for Mercurial are upstreamed already and are available in a standard Mercurial install.
I don't think anyone in this thread is CTO of a company size of Facebook. For a medium size company, you can easily use Mercurial or Perforce.
Re: Monorepo is great if you're really good
#135Earlier quoted context omitted.
We (~150 eng) build microservices in a polyglot environment (mostly Python, JS, and Go), all in a monorepo! We also build + deploy in containers with Jenkins, etc. The structure looks something like this: |-- third_party (third party libs) `-- python |-- libs (internally-written, common dependencies) `-- application_name |-- client (react app, connects to flask) |-- server (flask app, connects to services) `-- servic…
Organising apps by language seems weird...
Hierarchical layout demands choices be made, but there are advantages in grouping source packages by language:
1) can naturally reflect object packaging models of target language (eg python, jvm).
2) this can encourage reuse of packages across projects.
Re: Monorepo is great if you're really good
#136Earlier quoted context omitted.
The advantage of microkernels is that they can be extended with “untrusted” code like hardware drivers or file systems. This runs in user space and thus any bugs in such code will not crash the kernel process. So I agree with you that Linus is presenting a straw man and your comment shouldn’t have been downvoted.
> The advantage of microkernels is that they can be extended with “untrusted” code like hardware drivers or file systems. This runs in user space and thus any bugs in such code will not crash the kernel process. Did this advantage play out in practice? If your filesystem module goes down then every module that talks to the file system module needs to gracefully handle the failure or it will still effectively crash th…
My nvidia driver regularly hangs my system every ~90 minutes or so, so I can certainly empathize with the goals & vouch that they still have a role today.
Re: Monorepo is great if you're really good
#137Its funny and makes some good points but I don't think the distinction is between good and bad teams but whether or not you're 1 team or more. If one team has one application that is split across multiple repositories it can be a productivity boost and a simplification to unite them into a single repo with some single tools and norms. If you have two teams working primarily on two sets of repos and two different syst…
I think this only works if the teams are actually working on different products. If Team A and Team B are both writing to and reading from the same data stores, then I’d say they’re likely both working on the same product, and you have a multirepo.
Re: Monorepo is great if you're really good
#138Earlier quoted context omitted.
The advantage of microkernels is that they can be extended with “untrusted” code like hardware drivers or file systems. This runs in user space and thus any bugs in such code will not crash the kernel process. So I agree with you that Linus is presenting a straw man and your comment shouldn’t have been downvoted.
> The advantage of microkernels is that they can be extended with “untrusted” code like hardware drivers or file systems. This runs in user space and thus any bugs in such code will not crash the kernel process. Did this advantage play out in practice? If your filesystem module goes down then every module that talks to the file system module needs to gracefully handle the failure or it will still effectively crash th…
> Did this advantage play out in practice? If your filesystem module goes down then every module that talks to the file system module needs to gracefully handle the failure or it will still effectively crash the system.
If the file system process crashes then in theory the OS would simply relaunch it.
But your core services should be stable, it’s more about extensions, for example you may want to have virtual file systems (ftp, sshfs, etc.), which until FUSE wasn’t possible in the non-microkernel world.
As for how it played out in practice: I think microkernels lost early on because of performance and things like FUSE were created to allow the most obvious extension mechanisms for the otherwise non-extendable monolithic kernels.
Re: Monorepo is great if you're really good
#139Earlier quoted context omitted.
The advantage of microkernels is that they can be extended with “untrusted” code like hardware drivers or file systems. This runs in user space and thus any bugs in such code will not crash the kernel process. So I agree with you that Linus is presenting a straw man and your comment shouldn’t have been downvoted.
Please note that Linus wrote an operating system that in practice showed greater reliability than competing commercial microkernels. I do not believe that the principles that he came to believe in that process should be dismissed as straw man arguments.
> […] showed greater reliability than competing commercial microkernels
What is your basis for this claim?
I am only aware of QNX as a commercial microkernel (and real-time OS) and that is widely used in cars, medical devices, etc. with a strong reputation for reliability.
But for many tasks, Linux is good enough and free, which is hard to beat. But that does not mean that Linus is automatically correct in his statements.
Re: Monorepo is great if you're really good
#140My team just switched to a monorepo. It's been only a few weeks, so I can't claim any results yet, but we've lived w/ the pain of poly-repo for long enough that we were ready to invest in a single repo. We've spent a lot of time building and iterating a unified ci/cd environment to support the new repo. Previously each project had it's own test/deploy/build/publish story and usually it's own jenkins project. Now, eac…
> My team just switched to a monorepo.
I feel like this discussion is missing an appreciation for size/scope of repositories vs. size/scope of the organisation developing that software, with a pinch of appreciation for Conway's law.
If your team is a typical team of at most, say, 30 people, then maintaining 15 different repositories is clearly insane, but merging them into a single one likely doesn't truly deserve the moniker "monorepo", because it's just not that large (and varied in scope and purpose) of a project at the end of the day.
Think of it this way: the Linux kernel is certainly a larger project, but nobody thinks of it as a monorepo. Same thing goes for major software projects like Qt.