Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Google Stores Billions of Lines of Code in a Single Repository (2016)

cacm.acm.org

Post reply on HN
Print view View original

211–220 of 293 posts

Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)

#211

Post by erik_seaberg »

summerlight wrote: post #137
u801e wrote: post #122

Earlier quoted context omitted.

> So you never trample over the 0.1%. Instead you fix your code, or you fix their code for them -- which was probably due to your own bugs or undefined behavior in the first place. Or else you don't push. Given the size of a monrepo, is it possible to run the entire test suite in one's development environment, or do they have another endpoint to push to to run tests on a dedicated server?

Google has a CI infrastructure which runs most of the affected tests for each commit (which they call "CL") on thousand of machines in parallel. Though even for Google, running the entire test suite every time is prohibitively expensive so they have a way to merge and run multiple CLs in a single batch run every 3 hours, which is useful for testing a CL that may affect hundreds of thousands of build/test targets. If…

Google also regularly sees changelists break the world for hours, and each team has to sacrifice a member to serve as "build cop" and find the offending change and demand a rollback ASAFP.

Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)

#212

Post by Too »

Tloewald wrote: post #136
adamrt wrote: post #69

Earlier quoted context omitted.

We moved to a monorepo about 2 years ago and it has been nothing but success for us. We have quite a few projects but only 4 major applications. Maybe it is that a few of our projects intertwine a bit so making spanning changes in separate repositories was a pain. Doing separate PRs, etc. Now changes are more atomic. Our entire infrastructure can be brought up in development with a single docker-compose file and all…

My previous had a monorepo for the website and backend (but not the mobile apps) which was insane to work with (as a coder I had a dedicated 128 core box to work on, some engineers had more than one, less intense engineers shared one) and a substantial amount of my time was spent just finding code. I guess most engineers just end up working in some nook and so that searching code constantly becomes less of an issue (…

How is this

   company
     /ProjectA
       /.git
     /ProjectB
       /.git
easier to browse than this?

   company
     /.git
     /ProjectA
     /ProjectB
You still need to find the project A repo if you don't use monorepos. And even if you do use monorepos everything doesn't have to be one monoloithic build hogging down your IDE, you can still have microservices with the code for each hosted in the same repo. You seem to conflate monorepo with lots of other things.

Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)

#213

Post by mehh »

w_t_payne wrote: post #195
shados wrote: post #132

Earlier quoted context omitted.

> Is this a solved problem I don't mean that it's magical, just that it's not particularly sorcery. Instead of making a breaking change, add new method, deprecate old method. Update projects, then get rid of old deprecated method. Because they're distinct you can do this one by one so some project can reap the benefits without having to wait until all the problems are solved. Some people in this thread act like its f…

We have about 400 repos in a team of about 20 developers. We do have extensive tooling to help coordinate all of these, but configuration management is still by far the biggest engineering challenge that we face. I don't recall having such issues when I was working with Subversion and Perforce. On the other hand, not everything was rosy in the 'good old days': MS Source Safe was (by far) the worst VCS experience that…

Sounds excessive and beyond the norms of what most people would encounter in not having a mono repo i.e. the opposite extreme.

Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)

#214

Post by erik_seaberg »

hayleox wrote: post #171
paulie_a wrote: post #166

Im sure properly organized it's okay, but from what I've seen it's mediocre at best, especially with legacy/technical debt it's a huge mistake. Start breaking that repo apart, because it probably isn't very/hopefully depending on the debt that exists.

One of the big advantages of the monorepo is actually that it prevents technical debt from accumulating. If a change somewhere else breaks your code, you can't put off dealing with it -- you are forced to fix the issue immediately.

Tech debt is a useful tool and we shouldn't have zero tolerance. I can see wanting to deprecate old versions promptly, but I can't see instantly deprecating every old version with no workaround for mitigating emergencies.

Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)

#215

Post by Too »

bedros wrote: post #149
Tloewald wrote: post #130

Earlier quoted context omitted.

Can’t you create a branch and merge the two branches you are interested in into that?

my understanding, if you branch, you branch the entire repo, (not sure about some special case extensions ) if you have two projects stored in a single repo, you are forced to use whatever rev at for each project at a point of branch rev 5543 for example

Yep, and if you want to keep folders you didn't branch up to date with master you have to continuously rebase.

Not having it this way would be equivalent to having subrepo that refers to HEAD instead of a specific commit which is normally considered an big anti-pattern.

The best remedy is to not do branching like this in the first place, just try to stay on trunk all the time.

Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)

#216

Post by diminoten »

wirrbel wrote: post #153
hobls wrote: post #28

I feel terrible for anyone who sees this and thinks, “ah! I should move to a monorepo!” I’ve seen it several times, and the thing they all seem to overlook is that Google has THOUSANDS of hours of effort put into the tooling for their monorepo. Slapping lots of projects into a single git repo without investing in tooling will not be a pleasant experience.

Same line of thinking, just different conclusions. I feel terrible for anyone trying to run a company with open-source style independent repos. On a popular github project, you have MANY potential contributors that will tell you if a PR, or a release candidate break API compatibility, etc. There are thousands of hours in open source dedicated to fixing integration issues due to the (unavoidable) poly-repo situation.…

I'm honestly getting a little tired of the repetition this cycle causes. Monorepos are Wrong, "too many" repos are also Wrong, and everyone needs to realize that strawmaning the opposing side is really what wastes our time, not broken builds or waiting on dependencies to build.

Monorepo people ignore the learned lessons of those who came before us, and are trying to drag their teams back into a simpler time that, while nice, does not exist anymore. If you use any dependencies at all, you don't live in a monorepo world, and lying to yourself and your coworkers will only leave you confused and angry that your expectations are constantly not being met.

The solution isn't to split every single component into its own repo, but pretending like that's what anyone rational is proposing is not working with the best form of the argument. It's not always completely clear how to split up a growing codebase, but to claim that it's not usually worth splitting up is Wrong.

Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)

#217

Post by maccard »

jgibson wrote: post #194

Is it just me, or are a lot of people here conflating source control management and dependency management? The two don't have to be combined. For example, if you have Python Project X that depends on Python Project Y, you can either have them A) in different scm repos, with a requirements.txt link to a server that hosts the wheel artifact, B) have them in the same repo and refer to each other from source, or C) have…

I can't comment specifically on Google's tool, but I know it's based on perforce. perforce does have granular permissions - https://www.perforce.com/perforce/r15.1/manuals/p4sag/chapte...

Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)

#218

Post by Too »

ridiculous_fish wrote: post #13

> Google's monolithic software repository, which is used by 95% of its software developers worldwide, meets the definition of an ultra-large-scale4 system, providing evidence the single-source repository model can be scaled successfully This 95% number is the most surprising part of the article. That implies that the sum of engineers working on Android + Chrome + ChromeOS + all the Google X stuff + long tail of small…

That 95% is most likely more figurative than fact.

Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)

#219

Post by flukus »

vkou wrote: post #209
u801e wrote: post #119

Earlier quoted context omitted.

If the common code is a versioned package, then each of the 17 different projects could update their code to handle breaking changes in the common package independently and update the version dependency after thorough testing.

You can have versioned packages inside a mono-repository, too, though. /common_libs/foo_lib_v1.13/, /common_libs/foo_lib_v1.14/, etc.

By that point your creating micro-repositories in your mono-repository and getting the worst of both worlds.

Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)

#220

Post by justicezyx »

jgibson wrote: post #194

Is it just me, or are a lot of people here conflating source control management and dependency management? The two don't have to be combined. For example, if you have Python Project X that depends on Python Project Y, you can either have them A) in different scm repos, with a requirements.txt link to a server that hosts the wheel artifact, B) have them in the same repo and refer to each other from source, or C) have…

Single repo is one design that coherently addresses source control management and dependency management.

The key is to let the repo be a single comprehensive source of data for building arbitrary artifacts.

Post reply on HN