Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Monorepo: please do

medium.com

Post reply on HN
Print view View original

11–20 of 166 posts

Re: Monorepo: please do

#11

Post by jchw »

benmarten wrote: post #4
sharpercoder wrote: post #2

In my experience, this discussion gets convoluted by confusing modularity with monorepo . They are orthogonal to each other; you can have a very modular codebase in a monorepo but also a very coupled (non-modular) codebase with polyrepo. Though it's true that monorepos without proper discipline can tend towards coupling. Yet, when discussing mono vs poly, we should keep this in mind.

It's not, why do I have to checkout terabyte of code that I don't need, even if the code is modularized?

No need to checkout a terabyte of code. If your repo is scaling that high, you're going to want a VFS layer. Microsoft made a VFS layer for Git. As you might imagine, you simply grab files as needed, and your version control just deals with diffs for the most part. Google's own monorepo is proprietary but the Bazel build system is open source and would work great with a VCS hooked up with a VFS layer.

Re: Monorepo: please do

#12

Post by nkozyra »

erulabs wrote: post #6
benmarten wrote: post #4

Earlier quoted context omitted.

It's not, why do I have to checkout terabyte of code that I don't need, even if the code is modularized?

If a mono-repo has a terabyte of code, or if 10 small repos have 1/10th a terabyte each, what have you really gained? In any case, git LFS solves large file storage effectively, as do a number of other artifact storage solutions, and a repo with a terabyte of code is _not_ going to be trivially split apart, since it would be by a factor of thousands, the biggest codebase ever created by humankind.

> If a mono-repo has a terabyte of code, or if 10 small repos have 1/10th a terabyte each, what have you really gained?

If it's a small company where every developer touches every part of the application, sure. Taking the FAANG approach if you're not part of that acronym sounds like introducing inefficiency.

Re: Monorepo: please do

#13

Post by iseeyoubydesign »

what about the discussion of microservices? Why would you ever have these products separate by servers all in one repo? These technical constraints dont really help and are just another attempt to create a silver bullet.

Re: Monorepo: please do

#14

Post by skj »

benmarten wrote: post #4
sharpercoder wrote: post #2

In my experience, this discussion gets convoluted by confusing modularity with monorepo . They are orthogonal to each other; you can have a very modular codebase in a monorepo but also a very coupled (non-modular) codebase with polyrepo. Though it's true that monorepos without proper discipline can tend towards coupling. Yet, when discussing mono vs poly, we should keep this in mind.

It's not, why do I have to checkout terabyte of code that I don't need, even if the code is modularized?

Sounds like a tooling problem. We shouldn't use the current state of tooling as an excuse.

Re: Monorepo: please do

#15

Post by mmmeff »

Thank you so much for writing this. As someone who’s worked in the best and worst of these two words, the productivity gains are absolutely insane and the limitations, as stated by the author, are no more painful than limitations of federated/polyrepo code.

Fighting back against monorepo design is dangerous - embrace experimentation.

Re: Monorepo: please do

#16

Post by klodolph »

benmarten wrote: post #3

Please don't. It's just too slow and not efficient. Instead use common open source best practices of shared library architecture. Problem solved! Putting everything into one repo is just lack of organization and creates a huge mess.

I feel like you've really done no work supporting your argument there. "Slow and inefficient"... what, exactly, is slow and inefficient? Because there are plenty of things slow and inefficient about polyrepos.

I'd say that open-source best practices for shared libraries are appropriate if you're making an open-source shared library. However, these practices are inappropriate for internal libraries, proprietary libraries, and other use cases. In my experience, it's also far from "problem solved". You can point your finger at semantic versioning but in the meantime we go through hell and back with package managers trying to manage transitive library dependencies and it SUCKS. Why, for example, do you think people are fed up with NPM and created Yarn? Or why people constantly complain about Pip / Pipenv and the like? Why was the module system in Go 1.11 such a big deal? The answer is that it's hard to follow best practices for shared libraries, and even when you do follow best practices, you end up with mistakes or problems. These take engineering effort to solve. One of the solutions available is to use a monorepo, which doesn't magically solve all of your problems, it just solves certain problems while creating new problems. You have to weigh the pros and cons of the approaches.

In my experience, the many problems with polyrepos are mostly replaced with the relatively minor problems of VCS scalability and a poor branching story (mostly for long-running branches).

Re: Monorepo: please do

#17

Post by erulabs »

twblalock wrote: post #9
erulabs wrote: post #6

Earlier quoted context omitted.

If a mono-repo has a terabyte of code, or if 10 small repos have 1/10th a terabyte each, what have you really gained? In any case, git LFS solves large file storage effectively, as do a number of other artifact storage solutions, and a repo with a terabyte of code is _not_ going to be trivially split apart, since it would be by a factor of thousands, the biggest codebase ever created by humankind.

If I only need to check out one of the smaller repos then I've gained quite a lot in terms of download speed, storage size, etc. Git LFS adds a lot of complexity I'd rather avoid.

Sure but then you only have some small portion of the total infrastructure, which adds its own layer of complexity for the people reviewing your changes :P It's all trade offs, is all I'm saying - I honestly still can't decide between the two, although for all companies sub 20 people, I'd for sure stick with a single repo.

Re: Monorepo: please do

#18

Post by shados »

mmmeff wrote: post #15

Thank you so much for writing this. As someone who’s worked in the best and worst of these two words, the productivity gains are absolutely insane and the limitations, as stated by the author, are no more painful than limitations of federated/polyrepo code. Fighting back against monorepo design is dangerous - embrace experimentation.

> Fighting back against monorepo design is dangerous

What's dangerous about it? Monorepos have a lot of benefits, and should absolutely be considered. Maybe even by most. But right now in the community it's almost pushed as the "only true way with all benefits and no drawbacks", and that's absolutely not true. To the point the knowledge of why and how to poly repo is already starting to get lost.

That's dangerous.

Re: Monorepo: please do

#19

Post by totallysnowman »

I think that the reason of the argument is that both authors understand the definition of "large repository" very differently.

With 100 engineers a monorepo might seem a good idea. With 500 it becomes nearly impossible to do anything involving a build. Some isolation is needed.

Also from my experience many engineers just don't give a shit about architecture. They create entangled mess, that kind of works for the customer, and go home. Without some enforced isolation it is impossible to maintain it.

That being said I am more inclined to polyrepos.

Post reply on HN