Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Google Stores Billions of Lines of Code in a Single Repository (2016)

cacm.acm.org

Post reply on HN
Print view View original

281–290 of 293 posts

Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)

#281

Post by mlthoughts2018 »

gefh wrote: post #206
mlthoughts2018 wrote: post #93

One of my former managers had worked a long time at Google and was present for the advent of Google’s in-house tooling developed around their monorepo. His account was that it was basically accidental, at first resulting from short term fire drills, and then creating a snowball effect where the momentum of keeping things in the Perforce monorepo and building tooling around it just happened to be the local optimum, an…

Do you wonder whether he is a reliable narrator?

I don’t, but it’s fair to ask. He was unequivocally the best senior manager I’ve worked with. Extremely technically smart but skilled at letting people under him work autonomously, good communicator, cared a lot about pushing best practices past bureaucratic barriers.

His description of Google made it seem like it had the same dysfunction every place has. And the monorepo was a totally mundane, garden variety eyesore kind of in-house framework that you’ll find anywhere.

I think he recognized the usefulness of just working with it and picking battles. He was just dumbfounded that any outsider would see the monorepo project and think it possibly had any relevance for anyone else. It was just a Google-history-specific frankenstein sort of thing that got wrangled with tooling later. The supposed benefits are all just retrofitted on.

Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)

#282

Post by adrianN »

bunderbunder wrote: post #266
adrianN wrote: post #265

Earlier quoted context omitted.

It depends on the industry. In some places changing a dependency, no matter how trivial the change, entails a lot of work. Think for example about embedded systems where deploying is a lot harder than pushing a Docker image somewhere. It is often far cheaper to analyze whether the fixed bug can be triggered to avoid upgrading unless necessary.

In those situations, why not go ahead and keep the code up-to-date and consistent, and simply not deploy when you don't need to?

Because that costs money now that could be spent on something that actually produces a profit.

Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)

#283

Post by edejong »

bluejekyll wrote: post #248
thomaslee wrote: post #234

Earlier quoted context omitted.

Exactly this. Or at least it's a way this can be achieved, assuming solid testing & some tooling in the mix. For folks unfamiliar with it, the issue is something like: 1. You find a bug in a library A. 2. Libraries B, C and D depend on A. 3. B, C and D in turn are used by various applications. How do you fix a bug in A? Well, "normal" workflow would be something like: fix the bug in A, submit a PR, wait for a CI buil…

> So having run into this problem, folks like Google, Twitter, etc. use monorepos to help address some of this. I think you’re retroactively claiming that Google actively anticipated this in their choice at the beginning of using Perforce as an SCM. They may believe that it’s still the best option for them, but as I understand it, to make it work they bought a license to the Perforce source code forked it and practic…

Maybe Google’s choice for monorepo was pure chance. However, on many occasions the choice was challenged and these kinds of arguments were (successfully) made in order for it to stay.

Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)

#284

Post by Too »

flukus wrote: post #58
ebikelaw wrote: post #52

Earlier quoted context omitted.

See, this is where your argument broke down for me. Once you’ve decided there is some library of common code, and assuming you factor out that code into another repo, you’ve just lost your ability to easily make breaking changes to the common code, which is something trivially easy to do in a monorepo. Why would you want that? It seems to me that if you have multiple projects sharing a base of common code then a mono…

> you’ve just lost your ability to easily make breaking changes to the common code I haven't lost anything, I've gained the ability to make breaking changes because I don't have to update everything that breaks all at once. I don't have to do it at all because that's the job of the team responsible. With a monorepo what happens when their are 17 projects using the common code and I'm not familiar with 16 of them? Do…

If you are making a breaking change to a public interface you maintain, wouldn't you want to know how that interface is being used first, before justifying such a major breaking change? Not just change it for the sake of your own libraries internal convenience and hope that users of the library adopt. Since you know how the api is supposed to be used and all its best practices, understanding how to change the parts of the 16 projects using it should be quite easy, you shouldn't have to dig into the domain knowledge for all those projects.

Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)

#285

Post by Nihilartikel »

adrianN wrote: post #250
justicezyx wrote: post #220

Earlier quoted context omitted.

Single repo is one design that coherently addresses source control management and dependency management. The key is to let the repo be a single comprehensive source of data for building arbitrary artifacts.

A single repo makes it a bit tricky to use some library in version A for project X and version B for project Y.

If I recall, in Google's build system, a dependency in the source tree can be referenced at a commit ID, so you can actually have a dependency on an earlier version of artifacts in source control.

Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)

#286

Post by dgsb »

adamrt wrote: post #69
hobls wrote: post #28

I feel terrible for anyone who sees this and thinks, “ah! I should move to a monorepo!” I’ve seen it several times, and the thing they all seem to overlook is that Google has THOUSANDS of hours of effort put into the tooling for their monorepo. Slapping lots of projects into a single git repo without investing in tooling will not be a pleasant experience.

We moved to a monorepo about 2 years ago and it has been nothing but success for us. We have quite a few projects but only 4 major applications. Maybe it is that a few of our projects intertwine a bit so making spanning changes in separate repositories was a pain. Doing separate PRs, etc. Now changes are more atomic. Our entire infrastructure can be brought up in development with a single docker-compose file and all…

Just curious how small is small ? How many kloc ? How many people ?

Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)

#287

Post by favorited »

tiglionabbit wrote: post #108
favorited wrote: post #95

Earlier quoted context omitted.

Someone from FB did a really cringe-inducing presentation a few years ago about how "X can't handle our scale" (I think the predicate was iOS, but they went into IDEs and SCM systems). They had to pull the video and slides because it was so bad.

XCode also had issues working with large repos. Perhaps they were talking about that?

Responding late, so you might not see this. The thing that was so ridiculous to me was Facebook pretending that their app is somehow orders of magnitude more complicated than everyone else's.

They're whole schtick was "we're Facebook and we have unique scaling needs that no one else does," which makes absolutely no sense from the perspective of one user's content being rendered on their phone.

Plus the presenter was pretty smug, like all of this was good, when he wasn't convincing anyone that it was even necessary.

Found the slides: https://www.columbia.edu/~ng2573/zuggybuggy_is_2scale4ios.pd...

Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)

#288

Post by justicezyx »

Nihilartikel wrote: post #285
adrianN wrote: post #250

Earlier quoted context omitted.

A single repo makes it a bit tricky to use some library in version A for project X and version B for project Y.

If I recall, in Google's build system, a dependency in the source tree can be referenced at a commit ID, so you can actually have a dependency on an earlier version of artifacts in source control.

No, that's not true since at least 2013 (the year I joined Google).

Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)

#289

Post by krackers »

joe_fishfish wrote: post #244

This is probably a stupid question, but I couldn't find an answer. Does this mean Google keeps all of its different products in all their different languages and environments in one repo? So like, Android lives in the same repo as Gmail, which is the same repo as all the Waymo code and the Google search engine code as well? That seems insane to me.

Android & chromium are kept outside the monorepo

Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)

#290

Post by golangnews »

mr_tristan wrote: post #269
golangnews wrote: post #199

Earlier quoted context omitted.

What problems specifically did you see? Was this because the repo was too large? I understand at google scale you'd need lots of tooling but why at a smaller scslr of merging a dozen small repos?

The biggest problems are always cultural. Most monorepo workflows really reinforce constant integration, and once you have separate teams with separate managers, I've always witnessed constant conflict that ended up trying to establish spheres of control. It's bizarre - but it's something I've seen at pretty much every place I've worked at. With all that integration, your single CI toolchain is front and center since…

Thanks
Post reply on HN