Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Google Stores Billions of Lines of Code in a Single Repository (2016)

cacm.acm.org

Post reply on HN
Print view View original

181–190 of 293 posts

Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)

#181

Post by paulie_a »

hayleox wrote: post #171
paulie_a wrote: post #166

Im sure properly organized it's okay, but from what I've seen it's mediocre at best, especially with legacy/technical debt it's a huge mistake. Start breaking that repo apart, because it probably isn't very/hopefully depending on the debt that exists.

One of the big advantages of the monorepo is actually that it prevents technical debt from accumulating. If a change somewhere else breaks your code, you can't put off dealing with it -- you are forced to fix the issue immediately.

That makes a lot of sense and I definitely like the idea of that. Unfortunately unless you either spend a tremendous amount of effort in a legacy system to make that reality, or start with a new green field, it's not realistic day to day.

Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)

#182

Post by robaato »

makecheck wrote: post #120

This is clearly detrimental to external projects such as Go packaging, since their own developers will never be looking at dependency problems in the same way as outside groups. Monorepo also bugs me because there will always be some external package you need, and invariably it’s almost impossible to integrate due to years of colleagues making internal-only things assume everything imaginable about the structure and…

What about Android and 800-1,000 git repos?!

Have seen the pain trying to manage that across larger teams (e.g. thousands of devs) - and no the "repo" tool is not sufficient.

Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)

#183

Post by robaato »

axaxs wrote: post #174

Sorry, but as someone who has been in orgs that do both, mono repo is a mistake. Constant needs to pull unrelated changes before pushing, pipelines requiring to grab the whole repo for dependencies, etc. I understand the arguments for mono repo, but never think it's nothing that outweighs the cons.

Well those are issues around having a git mono repo - where the repo is the unit of change - you get it or you don't.

With mono repos such as SVN or Perforce you just work on whatever subset you want.

Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)

#184

Post by mbrock »

anyfoo wrote: post #107
whack wrote: post #87

Earlier quoted context omitted.

It took me a while to figure out that you're disagreeing with me, because your last paragraph is a perfect example of why monorepos are so dangerous. Imagine a tooling team on a different continent that makes some changes this afternoon. Like you said, their intent is just to add a new option, and it ought to have no extensional changes in behavior, but it still ends up behaving subtly different. The next morning, al…

> In a versioned world, you can still freeze your dependency at 1.324.5234, and migrate only when you want to, and when you're feeling confident about it. The correct course of action is to either reverse/fix the code change to the library you depend on, or if your code is clearly using the library wrong and can be easily fixed, to do that. Not to let the whole ecosystem slowly spiral out of control. Either way, the…

“Either way, the point is that it will force the issue to be resolved, quickly, and the code base to move forward.”

This is also known as continuous integration.

Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)

#185

Post by flukus »

rhencke wrote: post #88
dmoy wrote: post #68

Earlier quoted context omitted.

And gnucash, libreoffice, a couple copies of android, three other things that forked the linux kernel, and then all of apache to boot.

And we'll call it something crazy, like a Linux distribution!

A linux distro is a bunch of package metadata and binaries, they don't push the code of every package into one repository.

Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)

#186

Post by kqr »

013a wrote: post #173
hobls wrote: post #28

I feel terrible for anyone who sees this and thinks, “ah! I should move to a monorepo!” I’ve seen it several times, and the thing they all seem to overlook is that Google has THOUSANDS of hours of effort put into the tooling for their monorepo. Slapping lots of projects into a single git repo without investing in tooling will not be a pleasant experience.

I call this "Google Imposter Syndrome". Because Google (insert Facebook, Apple, Amazon, etc) has success with Monorepos (insert gRPC, Go, Kubernetes, React/Native, etc), it must be a great idea, we should do it. You see this everywhere . Also known as an Appeal to Authority. My personal opinion: very few companies will hit a point where sheer volume of code or code changes makes a monorepo unwieldy. Code volume is a…

Someone at some point said "Google may not be successful for the interview practises they use; they're big enough that they could very well be successful despite the interview practises they use."

It stuck with me, and is applicable to so many things. Including, maybe, this?

Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)

#187

Post by timkrueger »

We work with an monorepo since Septemeber 2017. I wrote about the migration:

https://timkrueger.me/a-maven-git-monorepo/

Our developers like it, because they can use 'mkdir' to create a new component, search threw the complete codebase with 'grep' and navigate with 'cd'.

Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)

#188

Post by [deleted] »

malkia wrote: post #178
lpghatguy wrote: post #165

Earlier quoted context omitted.

Most popular Git frontends (GitHub and GitLab too, I believe) let you link to commits with just the first 5-6 characters of the hash. I don't think that's much different to remember than a Perfore CL number.

To me the issue is when mentally trying to work with these numbers, P4 & G4's numbers increment, so I can tell which one came before the other - I can't do this with hashes. I'm sure I can get used to the other way, but this cannot easily be ignored.

[deleted]

Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)

#189

Post by timkrueger »

georgewfraser wrote: post #146
hobls wrote: post #28

I feel terrible for anyone who sees this and thinks, “ah! I should move to a monorepo!” I’ve seen it several times, and the thing they all seem to overlook is that Google has THOUSANDS of hours of effort put into the tooling for their monorepo. Slapping lots of projects into a single git repo without investing in tooling will not be a pleasant experience.

I kinda disagree, we’re a dev team of 30, 3.5 years in, 150k lines of code and we’ve always had a monorepo. We had to switch from maven to bazel after about 2 years because test times got out of control; bazel has been about 50% more annoying than maven but the incremental builds work perfectly.

Interesting. Do you have wrote something about that migration?

Re: Why Google Stores Billions of Lines of Code in a Single Repository (2016)

#190

Post by dguest »

glandium wrote: post #160
senozhatsky wrote: post #81

Well, it's not so uncommon. For instance, OpenBSD, NetBSD repos are sort of monolithic. And, believe it or not, there are some advantages. For instance, let's take a look at OpenBSD 5.5 [0] release notes: > OpenBSD is year 2038 ready and will run well > beyond Tue Jan 19 03:14:07 2038 UTC OpenBSD 5.5 was released on May 1, 2014. While Linux is still "not quite there yet" y2038-wise. y2038 is a very complex issue, whi…

The reason why y2038 problem looked so simple to OpenBSD has little to do with "monolithic repository" and everything to do with "happy to break kernel ABI compatibility". You're saying as much yourself. Monolithic repository might have been a tool that helped enforce it, but that's not what made it happen. It's the decision that ABI could be broken that did. And that's also why it hasn't happened in Linux yet. Even…

This is a very good point.

I work in an organization that just switched to monolithic and it's been going very well, with hundreds of active developers and millions of lines of code. But our developers are students or academics. As many as half don't understand the concept of an ABI. So the monorepo works quite well for us because rebuilding from scratch is something we do multiple times a day.

Post reply on HN