Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google stores billions of lines of code in a single repository (2016) [pdf]

dl.acm.org

Post reply on HN
Print view View original

181–190 of 209 posts

Re: Google stores billions of lines of code in a single repository (2016) [pdf]

#181

Post by rfoo »

pjmlp wrote: post #180
rfoo wrote: post #175

Earlier quoted context omitted.

There are languages / runtimes where there could not be two different versions of the same thing in one binary (and they eagerly fail at build time / immediately crash upon run). That is not the case for JavaScript, Rust, etc. But it is the case for C++, Java, Go, Python and more. Everyone claims different needs if they can. Nothing could be linked together anymore if you just let everyone use whatever they want. Or…

> But it is the case for C++, Java, Go, Python and more. It certainly isn't for Java, hence why multiple classloaders exist. For C and C++ it depends on the OS, on Windows (AIX, and similar OSes) this isn't an issue thanks to how symbol visibility works. Two different libraries are free to have whatever versions they feel like.

Thanks. I'm not familiar with Java. I thought multiple classloaders are more like dlmopen (which doesn't help much - symbol visibility is hard) cause I saw people struggling on classpath conflict etc.

Re: Google stores billions of lines of code in a single repository (2016) [pdf]

#182

Post by ashishb »

lopkeny12ko wrote: post #35

There's a lot of love for monorepos nowadays, but after more than a decade of writing software, I still strongly believe it is an antipattern. 1. The single version dependencies are asinine. We are migrating to a monorepo at work, and someone bumped the version of an open source JS package that introduced a regression. The next deploy took our service down. Monorepos mean loss of isolation of dependencies between ser…

I have worked at Google and have built multi-language outside Google.

1. Have some concept of visibility restriction e.g. Go language has internal package.

2. Ensure that every single package has a command to build the code.

3. Ensure that CI builds all the packages that changed our impacted by the change in a given pull request.

These three steps are mostly sufficient in having a monorepo. What you get in return is high code consistency and code visibility for the whole team.

Re: Google stores billions of lines of code in a single repository (2016) [pdf]

#183

Post by [deleted] »

rfoo wrote: post #174
dx034 wrote: post #162

Earlier quoted context omitted.

Sounds great to me because you are forced to delete code that's not in use anymore. Without the monorepo, that code would still be there with old libraries that are potentially insecure. Deleting code that is not being used anymore happens way too rarely in my opinion.

The downside is if a product no longer have maintainers you are now encouraged to shut it down, even if it still works and it doesn't cost much to run.

[deleted]

Re: Google stores billions of lines of code in a single repository (2016) [pdf]

#184

Post by KptMarchewa »

ywoski wrote: post #131
coryrc wrote: post #130

Earlier quoted context omitted.

> The monorepo works for Google Does it? Or is it stopping Google from supporting products which only make millions in revenue because of the massive burden of continually updating?

Oh geez, that's an entirely different can of worms that isn't related to the monorepo. Most products at Google are not dropped because the monorepo makes it difficult for them to support - and I'm not sure how it would or how you got to that association. Also, plenty of products that are killed are not in the monorepo. They are usually dropped due to a mix of things, but a big part is just better product management.

Fixed that:

>They are usually dropped due to a mix of things, but a big part is just worse product management.

Google is now at the point where their new projects fail (like Stadia) because they killed old products. Killing products has second-order effects.

Re: Google stores billions of lines of code in a single repository (2016) [pdf]

#185

Post by crubier »

rfoo wrote: post #174
dx034 wrote: post #162

Earlier quoted context omitted.

Sounds great to me because you are forced to delete code that's not in use anymore. Without the monorepo, that code would still be there with old libraries that are potentially insecure. Deleting code that is not being used anymore happens way too rarely in my opinion.

The downside is if a product no longer have maintainers you are now encouraged to shut it down, even if it still works and it doesn't cost much to run.

If a product non longer has maintainers, it's probably because it's not worth it for the company. So it makes sense to delete it, from the company point of view.

Re: Google stores billions of lines of code in a single repository (2016) [pdf]

#186

Post by Macha »

bananapub wrote: post #83
Macha wrote: post #66

Earlier quoted context omitted.

So what happens if search adopts your internal library and your update to it breaks search? Do you need to get someone from the search team to go investigate? How is that prioritised?

it's search's fault for locking down their code and so search gets to fix it.

And if they don't think it's important right now? Are other teams who need the change just blocked?

Re: Google stores billions of lines of code in a single repository (2016) [pdf]

#187

Post by dx034 »

rfoo wrote: post #174
dx034 wrote: post #162

Earlier quoted context omitted.

Sounds great to me because you are forced to delete code that's not in use anymore. Without the monorepo, that code would still be there with old libraries that are potentially insecure. Deleting code that is not being used anymore happens way too rarely in my opinion.

The downside is if a product no longer have maintainers you are now encouraged to shut it down, even if it still works and it doesn't cost much to run.

In that case the product should still have maintainers. Even if only part-time, no software project should be completely unsupervised.

Re: Google stores billions of lines of code in a single repository (2016) [pdf]

#188

Post by bananapub »

Macha wrote: post #186
bananapub wrote: post #83

Earlier quoted context omitted.

it's search's fault for locking down their code and so search gets to fix it.

And if they don't think it's important right now? Are other teams who need the change just blocked?

? then their code doesn't build anymore and they will have a very sad time.

Re: Google stores billions of lines of code in a single repository (2016) [pdf]

#189

Post by KptMarchewa »

zelphirkalt wrote: post #91
jmillikin wrote: post #46

Earlier quoted context omitted.

> The single version dependencies are asinine. We are migrating to > a monorepo at work, and someone bumped the version of an open > source JS package that introduced a regression. There's no requirement to have single versions of dependencies in a monorepo. Google allows[0] multiple versions of third-party dependencies such as jQuery or MySQL, and internal code is expected to specify which version it depends on. > I…

I guess the question then becomes: Is it worth all the extra tooling required to manage a monorepo properly?

There is a lot of extra tooling required to manage a large non-monorepo org too.

Re: Google stores billions of lines of code in a single repository (2016) [pdf]

#190

Post by spion »

rfoo wrote: post #175
spion wrote: post #168

Earlier quoted context omitted.

I don't think that's quite true. In my experience multi-repos have the edge here. If you have one key dependency update with a feature you need, but you need substantial code updates and 80 services depend on it, that may be impossible to pull off no matter what. Comparatively, upgrading one by one may not be easy, but at least its possible. The importance of everyone being on the same page with dependencies might ju…

There are languages / runtimes where there could not be two different versions of the same thing in one binary (and they eagerly fail at build time / immediately crash upon run). That is not the case for JavaScript, Rust, etc. But it is the case for C++, Java, Go, Python and more. Everyone claims different needs if they can. Nothing could be linked together anymore if you just let everyone use whatever they want. Or…

> There are languages / runtimes where there could not be two different versions of the same thing in one binary

But I'm not talking about one binary here. I'm talking about multiple, separate services.

Post reply on HN