Live data from Hacker News

Monorepos: Please don’t

medium.com

21–30 of 402 posts

Re: Monorepos: Please don’t

#21
Seems like the main point is that you'll still need to add additional tooling (search, local cloning, build, etc) to handle scaling, something you can do just as well with polyrepos. Conversely, for polyrepos, you can add tooling to fix issues with dependency management and multi-project changes/reviews. However, the author figures that monorepos engourage bad code culture and points out that Git is hard to build a monorepo on.

To me this message seems a bit shallow, of course we can build tooling to hide the fact that we have a polyrepo. Given well enough built tooling and consistent enough polyrepo structure (all using same VCS, all being linked from common tooling, following common coding standards and using the same build tooling, etc.) the distinction from having a monorepo is more of an implementation detail.

Given the choice between a consistent monorepo where everyone is running everything at HEAD and a polyrepo where each project have their own rules and there's no tooling to make a multi-project atomic change, I'd go for the former.

Given the choice between identical working environments but different underlying implementations I would go for whatever the tools team think is easier to maintain.

Re: Monorepos: Please don’t

#22
post #5

Alternate title: monorepos - ideal for teams under 100 devs

At the bank I used to work for, we had a monorepo with around 2500 developers committing in an average month. Total codebase was 35M LOC.

I gave a talk on it, "Python at Massive Scale", at PyData London 2018. Videos on YouTube/PyVideo.

Re: Monorepos: Please don’t

#23
post #10
post #5

Alternate title: monorepos - ideal for teams under 100 devs

At Uber, both of our iOS and Android teams are over 100 contributors each and we have a Monorepo for each app platform. I'm not on the ops team but being in a Monorepo here has been one of the best development experiences in my career.

FB has a monorepo for most of the known universe

Re: Monorepos: Please don’t

#24

I wish this had touched on polyrepos' ability to pin known-good versions of dependencies; that tends to be the Achilles' heel of monorepos.

NPM might not be the best package manager, but if you're using something like Lerna you can get the best of both worlds. Your local copy of an internal dependency can either be symlinked to the local source code, or a copy of the published package.

That makes it a lot easier to work on a package and its consumers at the same time.

Re: Monorepos: Please don’t

#25
> is there any real difference between checking out a portion of the tree via a VFS or checking out multiple repositories? There is no difference.

How big is your monorepo? Assume each line of code is a full 80 characters, stored via ASCII/UTF-8. That 67 million lines of code in 5GB. I can fit five of those on a Blu-ray.

> The end result is that the realities of build/deploy management at scale are largely identical whether using a monorepo or polyrepo.

True.

> It might be deployed over a period of hours, days, or months. Thus, modern developers must think about backwards compatibility in the wild.

Depends entire on the application. Lots of changes are deployed within short periods of time with low compatibility requirements.

> Downside 1: Tight coupling

Monorepos do often have tightly coupled software. Polyrepos also often have tightly coupled software. Polyrepos look more decoupled, but pragmatically I can't say I've noticed a much of difference.

> Downside 2: VCS scalability

I've also heard Twitter engineers complain about the VCS. But what is the scope of the author's discussion? 1,000 engineer orgs? Or 20 engineer orgs? Those are vastly different levels of engineering collaboration. I assume the article was not written to cover both of those. Or was it?

---

Ultimately, I think the author implicitly assumed a universe of discourse of gigantic repos with hundreds and hundreds of daily contributors.

When people talk about the spectrum of monorepo vs polyrepo architectures, that is very extreme. For example, last I knew, Uber has more repos than it did engineers. And I don't assume that "polyrepos" always means multiple repos per engineer.

Re: Monorepos: Please don’t

#26

I wish this had touched on polyrepos' ability to pin known-good versions of dependencies; that tends to be the Achilles' heel of monorepos.

That can be good, or that can be bad.

Someone makes an incompatible change, but you do not find out months later, because the client of that incompatibility is not using the latest versions. In the meantime, the development of the module and its downstreams has marched past any sort of easy resolution, and you essentially are now maintaining two different copies of your code.

Re: Monorepos: Please don’t

#27
post #23
post #10

Earlier quoted context omitted.

At Uber, both of our iOS and Android teams are over 100 contributors each and we have a Monorepo for each app platform. I'm not on the ops team but being in a Monorepo here has been one of the best development experiences in my career.

FB has a monorepo for most of the known universe

And Google has one that includes the multiverse.

Re: Monorepos: Please don’t

#28

I wish this had touched on polyrepos' ability to pin known-good versions of dependencies; that tends to be the Achilles' heel of monorepos.

NPM might not be the best package manager, but if you're using something like Lerna you can get the best of both worlds. Your local copy of an internal dependency can either be symlinked to the local source code, or a copy of the published package. That makes it a lot easier to work on a package and its consumers at the same time.

Oh God! I had forgotten how much frustration was in my old Python team until I checked the docs and discovered that you can make pip "install" your local copy of one repo as the dependency for another. The poor developer who prompted me to check was testing by doing CI builds and pulling down the new eggs.

Of course this isn't really a point against polyrepos, since it had a solution, but it's definitely something that I could imagine catching out lots of juniors.

Re: Monorepos: Please don’t

#29

My polyrepo cautionary tale: Two repos, one for fooclient, one for fooserver, talking to each other over protocol. Fooserver can do scary dangerous permanent things to company server instances, of which there are thousands. Fooserver sprouts a query syntax ("just do this for test servers A and B"), pushed to production. Fooclient sprouts code that relies on this, pushed to production. A bit later, Fooserver is rolled…

Ouch. I suppose the lesson is that a monorepo with both client and server being developed and tested together would have reduced such risk.

Versioning the client/server interface would've also reduced such risk.

Re: Monorepos: Please don’t

#30
Unless you are pure OSS or pure closed source - you end up with a poly-repo strategy regardless as you split open and closed code, suffering the annoyances of both systems.
Post reply on HN