I feel terrible for anyone who sees this and thinks, “ah! I should move to a monorepo!” I’ve seen it several times, and the thing they all seem to overlook is that Google has THOUSANDS of hours of effort put into the tooling for their monorepo. Slapping lots of projects into a single git repo without investing in tooling will not be a pleasant experience.
Why Google Stores Billions of Lines of Code in a Single Repository (2016)
161–170 of 293 posts
Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)
#162Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)
#163Earlier quoted context omitted.
> Like you said, their intent is just to add a new option, and it ought to have no extensional changes in behavior, but it still ends up behaving subtly different. The next morning, all your services end up broken as a result. Someone makes a commit to library code and production magically breaks? How does that happen?
> Someone makes a commit to library code and production magically breaks? How does that happen? With insufficient tests and broken release processes.
Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)
#164Earlier quoted context omitted.
And gnucash, libreoffice, a couple copies of android, three other things that forked the linux kernel, and then all of apache to boot.
And we'll call it something crazy, like a Linux distribution!
Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)
#165It should be noted that the monolithic model is somewhat encouraged by the client mapping system in Perforce, which was Google's first version control system so it is unclear to me if this was deliberate or just a side effect of the best VCS of the time. I also still have doubts around the value of a monorepo, in the article they claim it's valuable because you get: Unified versioning, one source of truth; Extensive…
Incrementally monolithic CL number is also useful. You can mark quite a lot of things with it - not only binary releases, but other developments too (configuration files, etc.). At the end your binary "version" comprises of main base CL + cherrypicked individual CL's - rather than branch with these fixes - I guess one can encode this too with git/hg - by using sha hashes - but this becomes much bigger in terms of inf…
Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)
#166Start breaking that repo apart, because it probably isn't very/hopefully depending on the debt that exists.
Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)
#167Earlier quoted context omitted.
There is a set of code changes locally and those changes are bundled off to the test server to run the full test suite? That's a branch. Now let's say I break a project sharing this code and because I'm not an expert in all 2 billion LoC and 3000 projects google is running I need to enlist some help in fixing what I broke. Presumably there is a way for the developers on that downstream project to pull in my change se…
Yeah that second thing doesn't exist. That first thing doesn't really exist the way you conceptualize either, I don't think.
You said the first thing doesn't exist? Do you not have local changes or are these changes not shared with the test/build server? Having a set of patches, code changes, whatever sounds like a branch to me, are you being too literal with the word branch?
For the second part what doesn't exist? Do you not make changes that breaks other peoples code? Do you not get them to help fix it? Can you not share your work in progress changes with others? Can you goes collaborate on changes at all?
Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)
#168I feel terrible for anyone who sees this and thinks, “ah! I should move to a monorepo!” I’ve seen it several times, and the thing they all seem to overlook is that Google has THOUSANDS of hours of effort put into the tooling for their monorepo. Slapping lots of projects into a single git repo without investing in tooling will not be a pleasant experience.
Both monorepo or "micro repo" end up falling apart at scale without some devops work involved. Either will work if you only have a few dozen projects. Neither will work once you hit 10s of millions of lines of code. But people seem to forget that it wasn't that long ago that git didn't exist, making multiple repos was a pain in the butt. Managing multiple repos locally was hell. Monorepos were the norm. Then as the s…
Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)
#169I feel terrible for anyone who sees this and thinks, “ah! I should move to a monorepo!” I’ve seen it several times, and the thing they all seem to overlook is that Google has THOUSANDS of hours of effort put into the tooling for their monorepo. Slapping lots of projects into a single git repo without investing in tooling will not be a pleasant experience.
Same line of thinking, just different conclusions. I feel terrible for anyone trying to run a company with open-source style independent repos. On a popular github project, you have MANY potential contributors that will tell you if a PR, or a release candidate break API compatibility, etc. There are thousands of hours in open source dedicated to fixing integration issues due to the (unavoidable) poly-repo situation.…
with that in mind, is monorepo is a universally good approach or is more dependent on good behavior of team members than polyrepo?
Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)
#170Earlier quoted context omitted.
my understanding, if you branch, you branch the entire repo, (not sure about some special case extensions ) if you have two projects stored in a single repo, you are forced to use whatever rev at for each project at a point of branch rev 5543 for example
In Perforce, which is more or less what Google is using, you can branch any directory within the repo. (You would never branch the whole repo; that makes no sense.) So if you wanted to construct a directory with one version of one subdirectory, and a different version of another subdirectory, that's quite straightforward.