Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Google Stores Billions of Lines of Code in a Single Repository (2016)

cacm.acm.org

Post reply on HN
Print view View original

41–50 of 293 posts

Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)

#41

Post by acomjean »

hobls wrote: post #28

I feel terrible for anyone who sees this and thinks, “ah! I should move to a monorepo!” I’ve seen it several times, and the thing they all seem to overlook is that Google has THOUSANDS of hours of effort put into the tooling for their monorepo. Slapping lots of projects into a single git repo without investing in tooling will not be a pleasant experience.

But what are the alternatives to the monorepo in git?

All the ways of splitting code up and deploying multiple git repos for one project seem terrible.

Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)

#42

Post by mr_tristan »

hobls wrote: post #28

I feel terrible for anyone who sees this and thinks, “ah! I should move to a monorepo!” I’ve seen it several times, and the thing they all seem to overlook is that Google has THOUSANDS of hours of effort put into the tooling for their monorepo. Slapping lots of projects into a single git repo without investing in tooling will not be a pleasant experience.

I sense that Google invests much more in it's infrastructure then most companies make in revenue.

I've worked with monorepos, and I'd be loathe to recommend it as well; the combination of culture shift and tooling it takes to keep a monorepo system running makes most CD processes you see today look like child's play.

There is a lot of very good free software that supports most of the open source approach to CD these days; but very, very little freely available monorepo tooling. Just check out https://github.com/korfuri/awesome-monorepo - it's a quick read. I haven't found many other notably superior compilations. Compared with available OSS workflows and tooling, it's rather sparse, filled with bespoke approaches everywhere.

Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)

#43

Post by forrestthewoods »

acomjean wrote: post #41
hobls wrote: post #28

I feel terrible for anyone who sees this and thinks, “ah! I should move to a monorepo!” I’ve seen it several times, and the thing they all seem to overlook is that Google has THOUSANDS of hours of effort put into the tooling for their monorepo. Slapping lots of projects into a single git repo without investing in tooling will not be a pleasant experience.

But what are the alternatives to the monorepo in git? All the ways of splitting code up and deploying multiple git repos for one project seem terrible.

> what are the alternatives to the monorepo in git?

A monorepo in Perforce!

Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)

#44

Post by threeseed »

acomjean wrote: post #41
hobls wrote: post #28

I feel terrible for anyone who sees this and thinks, “ah! I should move to a monorepo!” I’ve seen it several times, and the thing they all seem to overlook is that Google has THOUSANDS of hours of effort put into the tooling for their monorepo. Slapping lots of projects into a single git repo without investing in tooling will not be a pleasant experience.

But what are the alternatives to the monorepo in git? All the ways of splitting code up and deploying multiple git repos for one project seem terrible.

I don't understand why people are against this. You can have per repo branches/tags, the history is clean and relevant, it's easy to triage breakages, easy for different apps to have different versions of code etc. Plus for CI/CD it's trivial to just have one Jenkins jobs per repo as well and simple Git commit triggers.

The entire programming world revolves around libraries and yet when it comes to our own code we are afraid of them ? Strange.

Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)

#45

Post by threeseed »

nine_k wrote: post #36
hobls wrote: post #28

I feel terrible for anyone who sees this and thinks, “ah! I should move to a monorepo!” I’ve seen it several times, and the thing they all seem to overlook is that Google has THOUSANDS of hours of effort put into the tooling for their monorepo. Slapping lots of projects into a single git repo without investing in tooling will not be a pleasant experience.

Most folks who consider a monorepo don't have billions of lines of code, and often not even millions. Linux kernel is a monorepo.

Linux kernel is one functional piece of work though.

Imagine if we combined KDE, Gnome, Linux Kernel, ZFS etc all in the one monorepo.

Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)

#46

Post by reificator »

acomjean wrote: post #41
hobls wrote: post #28

I feel terrible for anyone who sees this and thinks, “ah! I should move to a monorepo!” I’ve seen it several times, and the thing they all seem to overlook is that Google has THOUSANDS of hours of effort put into the tooling for their monorepo. Slapping lots of projects into a single git repo without investing in tooling will not be a pleasant experience.

But what are the alternatives to the monorepo in git? All the ways of splitting code up and deploying multiple git repos for one project seem terrible.

> multiple git repos for one project

If it's one project, it's not a monorepo. It's a repo.

Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)

#48

Post by flukus »

acomjean wrote: post #41
hobls wrote: post #28

I feel terrible for anyone who sees this and thinks, “ah! I should move to a monorepo!” I’ve seen it several times, and the thing they all seem to overlook is that Google has THOUSANDS of hours of effort put into the tooling for their monorepo. Slapping lots of projects into a single git repo without investing in tooling will not be a pleasant experience.

But what are the alternatives to the monorepo in git? All the ways of splitting code up and deploying multiple git repos for one project seem terrible.

> All the ways of splitting code up and deploying multiple git repos for one project seem terrible.

Of course they are, git isn't the tool for this. You don't want multiple repos for a single project, you want one per project (this is not a monorepo). If there are things like code common to multiple projects then they are their own project with their own repo and release schedule, releases go into some sort of package manager (even if it's just the file system) and the projects depending on that common code update as they go.

Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)

#49

Post by hobls »

rco8786 wrote: post #34
hobls wrote: post #28

I feel terrible for anyone who sees this and thinks, “ah! I should move to a monorepo!” I’ve seen it several times, and the thing they all seem to overlook is that Google has THOUSANDS of hours of effort put into the tooling for their monorepo. Slapping lots of projects into a single git repo without investing in tooling will not be a pleasant experience.

Are you suggesting that there’s a solution to managing large amounts of code that doesn’t involve large amounts of tooling?

There’s an important distinction between lots of code and lots of projects. I agree; if you have a ton of code you’d better invest in tooling. But if you just have several normally sized projects, a monorepo can make your life much more difficult than simply using several repos.

Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)

#50

Post by jorblumesea »

fastball wrote: post #39
jorblumesea wrote: post #33

Is this really relevant for anyone except for "google scale" companies? For most teams, managing 30-40 services backed by git repos isn't a huge task and doesn't cause many problems. Is there mature tooling that helps teams manage this, or is this proprietary google magic tooling?

Most teams can probably get by with much fewer than 30-40 services. Unless you have 30-40 groups within your team.

Even if they had that, managing the contract between and in a small group isn't super difficult.
Post reply on HN