Live data from Hacker News

Google stores billions of lines of code in a single repository (2016) [pdf]

dl.acm.org

11–20 of 209 posts

Re: Google stores billions of lines of code in a single repository (2016) [pdf]

#11
Related:

Why Google Stores Billions of Lines of Code in a Single Repository (2016) - https://news.ycombinator.com/item?id=22019827 - Jan 2020 (121 comments)

Why Google Stores Billions of Lines of Code in a Single Repository (2016) - https://news.ycombinator.com/item?id=17605371 - July 2018 (281 comments)

Why Google stores billions of lines of code in a single repository (2016) - https://news.ycombinator.com/item?id=15889148 - Dec 2017 (298 comments)

Why Google Stores Billions of Lines of Code in a Single Repository - https://news.ycombinator.com/item?id=11991479 - June 2016 (218 comments)

Re: Google stores billions of lines of code in a single repository (2016) [pdf]

#15
post #10

Monorepos are great... but only if you can invest in the tooling scale to handle them, and most companies can't invest in that like Google can. Hyrum Wright class tooling experts don't grow on trees. A good article to reference when this topic gets raised: http://yosefk.com/blog/dont-ask-if-a-monorepo-is-good-for-yo...

You can get better tools now though, like Turbo Repo or NX. They don’t require the same level of investment as Bazel but they don’t always have the same hermetic build guarantees, though for most it’s “good enough”.

Re: Google stores billions of lines of code in a single repository (2016) [pdf]

#18
post #8
post #6

I really wish they would make this tech available via gcloud. Seems like it would be very popular and a great way to attract other gcloud business away from MS/GitHub which scales horribly.

They tried that by making a bit available with a remote cloud builder for Bazel. It failed for some reason and they pulled it. I think building something that scales for one big repo is just a completely different problem than making it scale for a lot of small repos.

Long term projects like this don't get any attention because the chance of getting a promotion from it are almost nil.

And after the layoffs, it's pretty clear that no matter how hard you work, you can get fired so what's the point in dedicating your career to something like this?

Re: Google stores billions of lines of code in a single repository (2016) [pdf]

#19

iOS and Windows are “monorepos” too. The software is built daily, and everyone must be on the same version of every library. Under the hood there are a bunch of repos, and there are exceptions, but largely operates as a monorepo.

Is this still the case for Windows? I remember hearing something like this when I was getting my BCompSci, but I assumed it must have changed since then.

Re: Google stores billions of lines of code in a single repository (2016) [pdf]

#20
post #10

Monorepos are great... but only if you can invest in the tooling scale to handle them, and most companies can't invest in that like Google can. Hyrum Wright class tooling experts don't grow on trees. A good article to reference when this topic gets raised: http://yosefk.com/blog/dont-ask-if-a-monorepo-is-good-for-yo...

You don't need google scale tooling to work with a mono repo until you are actually at google scale. Gluing together a bunch of separate repos isn't exactly free either. See, for example, the complicated disaster Amazon has with brazil.

In the limit, there are only two options:

  1. All code lives one repo
  2. Every function/class/entity lives in its own repo
with a third state in between

  3. You accept code duplication
This compromise state where some code duplication is (maybe implicitly) acceptable is what most people have in mind with a poly-repo.

The problem though is that (3) is not a stable equilibrium. Most engineers have such a kneejerk reaction against code duplication that (3) is practically untenable. Even if your engineers are more reasonable, (3) style compromise means they constantly have to decide "should this code from package A be duplicated in package B, or split off into a new smaller package C, which A and B depend on". People will never agree on the right answer, which generates discussion and wastes engineering time. In my experience, the trend is almost never to combine repos, but always to generate more and more repos.

The limiting case of a mono repo (which is basically it's natural state) is far more palatable than the limiting case of poly-repo.

Post reply on HN