Live data from Hacker News

Google stores billions of lines of code in a single repository (2016) [pdf]

dl.acm.org

201–209 of 209 posts

Re: Google stores billions of lines of code in a single repository (2016) [pdf]

#201
post #196
post #118

Earlier quoted context omitted.

I want to think they have. But... this is also why they kill older products. The cost of keeping the lights on is greatly elevated when keeping the lights on means keeping up with the latest codes. This is absolutely no different from buildings. If you had to keep every building up to date with the latest building codes, you would tear them down way way way more often.

> this is also why they kill older products. The cost of keeping the lights on is greatly elevated when keeping the lights on means keeping up with the latest codes. This is a really good point and I think accurate when it comes to smaller Google endeavors. I don't think this killed Stadia, for example, but maybe Google Trips (an amazing service that I don't think many folks used and likely had few development resour…

Yeah, I would not mean this to include Stadia. That said, if it adds costs to the smaller Trips and such, it has to add cost to the larger things, too. That is, if it makes the cheap things expensive, it probably makes the expensive things even more so.

Re: Google stores billions of lines of code in a single repository (2016) [pdf]

#202
post #193
post #190

Earlier quoted context omitted.

> There are languages / runtimes where there could not be two different versions of the same thing in one binary But I'm not talking about one binary here. I'm talking about multiple, separate services.

How many internal libraries does your "separate services" contain? You service A depends on library alpha@1, your service B depends on library alpha@2. All happy now. Introduce another layer, your service A depends on library alpha@1, beta@1, and alpha@1 depends on gamma@1, beta@1 depends on gamma@2, what to do now? It does not even matter how many services you have now. With Javascript it does not apply, alpha@1 can…

> How many internal libraries does your "separate services" contain? You service A depends on library alpha@1, your service B depends on library alpha@2. All happy now. Introduce another layer, your service A depends on library alpha@1, beta@1, and alpha@1 depends on gamma@1, beta@1 depends on gamma@2, what to do now? It does not even matter how many services you have now.

Got several thoughts on this one. First, lets look at how bad the issue really is:

To start using beta@1 you need to upgrade alpha@1 to alpha@2 that depends on gamma@2. What's the problem with that?

The same situation can arise with 3rd party dependencies, except there its much worse: you have zero control over those. Here you do have the control.

Now lets look at what this situation looks like in a monorepo: you can't even introduce gamma@2 and make beta@1 at all without

1. upgrading alpha@1 to alpha@2

2. upgrading all services that depend on alpha@2

3. upgrading all libraries that depend on gamma@2

4. upgrading all services that depend on gamma@2, if any

So you might even estimate that the cost of developing beta@2 is not worth it at all. Instead of quasi-dependency-hell ("quasi" because your company still controlls all those libraries and has power to fix the issue unlike real dependency hell) you have a real stagnation hell due to a thousand papercuts

My second comment is about building deep "layers" of internal dependencies - I would recommend avoiding it for as long as possible. Not just because of versioning, but because that itself causes stagnation. The more things depend on a piece of code, the harder it is to manage it effectively or to make any changes to it. The deeper the dependency tree is, the harder it is to reason about the effect of changes. So you better be very certain about their design / API surface and abstraction before building such dependencies yourself.

Major version bumps of foundational library dependencies is an indication that you originally had the wrong abstraction. No matter how you organize your code in your repos, its going to be a problem. (Incidentally, this is also why despite the flexibility of node_modules, we still have JS fatigue. At least with internal dependencies we can work to avoid such churn.) It should still be easier with separate services, however, as you can do it more gradually.

Last note on left-pad and similar libraries. They are a different beast. They have a clear scope, small size and most importantly, zero probability of needing any interface changes (very low probability of any code changes as well). That makes them a less risky proposition (assuming of course they cannot be deleted)

Those are my (hopefully nuanced) 2 cents.

Re: Google stores billions of lines of code in a single repository (2016) [pdf]

#203
post #202
post #193

Earlier quoted context omitted.

How many internal libraries does your "separate services" contain? You service A depends on library alpha@1, your service B depends on library alpha@2. All happy now. Introduce another layer, your service A depends on library alpha@1, beta@1, and alpha@1 depends on gamma@1, beta@1 depends on gamma@2, what to do now? It does not even matter how many services you have now. With Javascript it does not apply, alpha@1 can…

> How many internal libraries does your "separate services" contain? You service A depends on library alpha@1, your service B depends on library alpha@2. All happy now. Introduce another layer, your service A depends on library alpha@1, beta@1, and alpha@1 depends on gamma@1, beta@1 depends on gamma@2, what to do now? It does not even matter how many services you have now. Got several thoughts on this one. First, let…

Note: steps 1-4 will likely be done in somewhat reverse order (3,4 first, then 1,2, then building `beta`)

Re: Google stores billions of lines of code in a single repository (2016) [pdf]

#204
post #202
post #193

Earlier quoted context omitted.

How many internal libraries does your "separate services" contain? You service A depends on library alpha@1, your service B depends on library alpha@2. All happy now. Introduce another layer, your service A depends on library alpha@1, beta@1, and alpha@1 depends on gamma@1, beta@1 depends on gamma@2, what to do now? It does not even matter how many services you have now. With Javascript it does not apply, alpha@1 can…

> How many internal libraries does your "separate services" contain? You service A depends on library alpha@1, your service B depends on library alpha@2. All happy now. Introduce another layer, your service A depends on library alpha@1, beta@1, and alpha@1 depends on gamma@1, beta@1 depends on gamma@2, what to do now? It does not even matter how many services you have now. Got several thoughts on this one. First, let…

> To start using beta@1 you need to upgrade alpha@1 to alpha@2 that depends on gamma@2. What's the problem with that?

The problem is the team maintaining alpha does not want to upgrade to gamma@2 because it's an extra burden for them, and they don't have an immediate need.

The debate is not about teams owning separate services, it's about teams owning libraries.

Re: Google stores billions of lines of code in a single repository (2016) [pdf]

#205
post #204
post #202

Earlier quoted context omitted.

> How many internal libraries does your "separate services" contain? You service A depends on library alpha@1, your service B depends on library alpha@2. All happy now. Introduce another layer, your service A depends on library alpha@1, beta@1, and alpha@1 depends on gamma@1, beta@1 depends on gamma@2, what to do now? It does not even matter how many services you have now. Got several thoughts on this one. First, let…

> To start using beta@1 you need to upgrade alpha@1 to alpha@2 that depends on gamma@2. What's the problem with that? The problem is the team maintaining alpha does not want to upgrade to gamma@2 because it's an extra burden for them, and they don't have an immediate need. The debate is not about teams owning separate services, it's about teams owning libraries.

I'm assuming a customer-driven culture where you work for your customers needs. In the case of libraries, teams using the libraries are customers. If you're the maintainer of alpha and your customer needs beta, your customer needs you to upgrade to gamma.

Re: Google stores billions of lines of code in a single repository (2016) [pdf]

#206
post #205
post #204

Earlier quoted context omitted.

> To start using beta@1 you need to upgrade alpha@1 to alpha@2 that depends on gamma@2. What's the problem with that? The problem is the team maintaining alpha does not want to upgrade to gamma@2 because it's an extra burden for them, and they don't have an immediate need. The debate is not about teams owning separate services, it's about teams owning libraries.

I'm assuming a customer-driven culture where you work for your customers needs. In the case of libraries, teams using the libraries are customers. If you're the maintainer of alpha and your customer needs beta, your customer needs you to upgrade to gamma.

But then another customer still wants gamma@1, they are allowed to do that! But they also want your new features. So now you have to maintain two branches, which I hope we can agree: it is an extra burden.

This is unavoidable if we are talking about FOSS, people should be able to do whatever they want, and they do. A company has an advantage here: you can install company-wide rules and culture to make sure people don't do this. Which, in this case, happens to be: let's keep a single version of everything unless you have really good reasons.

Re: Google stores billions of lines of code in a single repository (2016) [pdf]

#207
post #206
post #205

Earlier quoted context omitted.

I'm assuming a customer-driven culture where you work for your customers needs. In the case of libraries, teams using the libraries are customers. If you're the maintainer of alpha and your customer needs beta, your customer needs you to upgrade to gamma.

But then another customer still wants gamma@1, they are allowed to do that! But they also want your new features. So now you have to maintain two branches, which I hope we can agree: it is an extra burden. This is unavoidable if we are talking about FOSS, people should be able to do whatever they want, and they do. A company has an advantage here: you can install company-wide rules and culture to make sure people don…

> But then another customer still wants gamma@1, they are allowed to do that! But they also want your new features.

In this case, you still have the option of working with them to help them migrate to gamma@2, if the cost of maintaining gamma@1 is indeed too high and would negatively impact you in serving other customers. This was the original premise, wasn't it - upgrading all your dependants when you upgrade your library? That's still an option. The point is - you have more choices. And you can also help customers one by one - you don't have to do it all at once

I will agree though, restricting choices helps when the company is finding difficulty in aligning incentives through communication. But you do give up a lot for it - including ability to move fast and avoid stagnation.

Re: Google stores billions of lines of code in a single repository (2016) [pdf]

#208
post #207
post #206

Earlier quoted context omitted.

But then another customer still wants gamma@1, they are allowed to do that! But they also want your new features. So now you have to maintain two branches, which I hope we can agree: it is an extra burden. This is unavoidable if we are talking about FOSS, people should be able to do whatever they want, and they do. A company has an advantage here: you can install company-wide rules and culture to make sure people don…

> But then another customer still wants gamma@1, they are allowed to do that! But they also want your new features. In this case, you still have the option of working with them to help them migrate to gamma@2, if the cost of maintaining gamma@1 is indeed too high and would negatively impact you in serving other customers. This was the original premise, wasn't it - upgrading all your dependants when you upgrade your l…

Ah, ok. I see where you are coming from :)

From what I saw I'd say it's exactly opposite: allowing multiple versions actually means "make teams able to choose for stagnation". And because we are lazy, we certainly do! There is a non-trivial amount of people who believes "if it ain't broken don't fix it". I can work with them to migrate them over, but they might not want to do so! In this case, a hard "bump versions or die" rule is a must.

Maybe if you work in a small group of great engineers you don't need to set such rules and you can move even faster, but I unfortunately haven't found such a workplace :(

> you don't have to do it all at once

Yes. Nobody should do it all at once. Making "bump versions or die" compatible with incremental adoption is slightly harder (see sibling threads for how it's done). Still worth it I'd argue.

Re: Google stores billions of lines of code in a single repository (2016) [pdf]

#209
post #191

Earlier quoted context omitted.

? then their code doesn't build anymore and they will have a very sad time.

But I thought the precommit hooks would block this merge if it breaks search?

There’s no merge or precommit hooks or blocking.

If they chose to make their code invisible to me then I can’t fix it.

Post reply on HN