Earlier quoted context omitted.
How do you deal with wanting to see the history, graph etc of just one sub-project? Does the tooling handle this?
git log accomplishes this already.
Google uses piper and perforce (well, g4) before that
Google stores billions of lines of code in a single repository (2016) [pdf]
61–70 of 209 posts
Re: Google stores billions of lines of code in a single repository (2016) [pdf]
#62Earlier quoted context omitted.
> The single version dependencies are asinine. We are migrating to > a monorepo at work, and someone bumped the version of an open > source JS package that introduced a regression. There's no requirement to have single versions of dependencies in a monorepo. Google allows[0] multiple versions of third-party dependencies such as jQuery or MySQL, and internal code is expected to specify which version it depends on. > I…
> There's no requirement to have single versions of dependencies in a monorepo. Google allows[0] multiple versions of third-party dependencies such as jQuery or MySQL, and internal code is expected to specify which version it depends on. Sure, but this is unsustainable. If service Foo depends on myjslib v3.0.0, but service Bar needs to pull in myjslib v3.1.0, in order to make sure Foo is entirely unchanged, you'd hav…
> Sure, but this is unsustainable.
Not exactly unsustainable considering Google has been very successful with this approach!
Re: Google stores billions of lines of code in a single repository (2016) [pdf]
#63Earlier quoted context omitted.
> any breaking changes should be dealt with in the same change as the version bump Does this mean that some things will never get updated, as the effort required is impossibly high?
No you use automated systems to do the change. https://mobile.twitter.com/obeattie/status/10804969557537505...
Even so, the cost is often outrageously high.
No to mention that if you're the first team to import a third_party library, you own it and other teams can add arbitrary cost to you updating it. You have to be very aggressive with visibility and SLAs to work around this.
Re: Google stores billions of lines of code in a single repository (2016) [pdf]
#64I've never experienced a monorepo like Googles. How does it work? Are Chrome and Gmail in the same repo? I assume they're built separately and pushing code to one doesn't affect the other.
No, Chrome and Gmail are in different monorepos.
>How does it work?
Different projects are in different folders instead of different repos.
>I assume they're built separately and pushing code to one doesn't affect the other.
Yes, building or testing something only builds its dependencies.
Re: Google stores billions of lines of code in a single repository (2016) [pdf]
#65Earlier quoted context omitted.
> The single version dependencies are asinine. We are migrating to > a monorepo at work, and someone bumped the version of an open > source JS package that introduced a regression. There's no requirement to have single versions of dependencies in a monorepo. Google allows[0] multiple versions of third-party dependencies such as jQuery or MySQL, and internal code is expected to specify which version it depends on. > I…
> There's no requirement to have single versions of dependencies in a monorepo. Google allows[0] multiple versions of third-party dependencies such as jQuery or MySQL, and internal code is expected to specify which version it depends on. Sure, but this is unsustainable. If service Foo depends on myjslib v3.0.0, but service Bar needs to pull in myjslib v3.1.0, in order to make sure Foo is entirely unchanged, you'd hav…
> Sure, but this is unsustainable. [...] After two years you'd have
> 10 unique dependencies for 10 versions of myjslib in the monorepo.
Note that this need for a policy exists for both mono-repo and multi-repo worlds. Handling of third-party dependencies ought to be independent of how the version control repository is structured.
> At this point you've basically replicated the dependency semantics of
> a multi-repo world to a monorepo, with extra cruft. This problem is
> already implicitly solved in a multi-repo world because each service
> simply declares its own dependencies.
That's why monorepos exist. They're not a way to force people to upgrade dependencies, and they aren't a get-out-of-jail-free card for thinking about inter-project dependencies. A monorepo lets you have a linear view of code history.
Phrased differently: many people approach monorepos as a way to force their view of dependency management on other people in their organization. The successful users of monorepos (including Google) take great efforts to let separate projects in the same repo operate independently.
Re: Google stores billions of lines of code in a single repository (2016) [pdf]
#66> Google’s codebase is shared by more [...] than 25,000 Google software develop- ers from dozens of offices in countries around the world. > Access to the whole codebase encourages extensive code sharing and reuse [...] Doesn't this strategy result in a great risk of massive code leaks from rogue employees? Even if read access are logged and the culprit found, it's too late once it's been published.
Despite almost everything being in one big repo, it has silos. Not everyone has read access to everything. Some code, like the important bits of Search, is only available on a need-to-know basis.
So what happens if search adopts your internal library and your update to it breaks search? Do you need to get someone from the search team to go investigate? How is that prioritised?
Re: Google stores billions of lines of code in a single repository (2016) [pdf]
#67Earlier quoted context omitted.
Not updating old libraries is how you end up getting known security vulns years after they are patched.
You should ask your colleagues who work in critical industries like banking and healthcare how much of their software stack depends on things that haven't been patched in more than 20 years ;)
Healthcare developer here, developing for German market, we've used Java preview features and unstable React versions many times before. And we literally have two different roles on our team for upgrading vulnerable dependencies whenever we get an alert.
Re: Google stores billions of lines of code in a single repository (2016) [pdf]
#68Earlier quoted context omitted.
Not updating old libraries is how you end up getting known security vulns years after they are patched.
You should ask your colleagues who work in critical industries like banking and healthcare how much of their software stack depends on things that haven't been patched in more than 20 years ;)
I've worked in healthcare, I find the software practices absolutely atrocious. The consequence of this has been ransomware attacks:
https://www.usnews.com/news/health-news/articles/2022-10-10/...
https://www.bloomberg.com/news/features/2023-02-03/ireland-h...
https://floridapolitics.com/archives/585686-tallahassee-memo...
https://www.oceancitytoday.com/news/atlantic-general-hospita...
Re: Google stores billions of lines of code in a single repository (2016) [pdf]
#69Monorepos are great... but only if you can invest in the tooling scale to handle them, and most companies can't invest in that like Google can. Hyrum Wright class tooling experts don't grow on trees. A good article to reference when this topic gets raised: http://yosefk.com/blog/dont-ask-if-a-monorepo-is-good-for-yo...
You can get better tools now though, like Turbo Repo or NX. They don’t require the same level of investment as Bazel but they don’t always have the same hermetic build guarantees, though for most it’s “good enough”.
Build in docker.
Re: Google stores billions of lines of code in a single repository (2016) [pdf]
#70Monorepos are great... but only if you can invest in the tooling scale to handle them, and most companies can't invest in that like Google can. Hyrum Wright class tooling experts don't grow on trees. A good article to reference when this topic gets raised: http://yosefk.com/blog/dont-ask-if-a-monorepo-is-good-for-yo...
Why are monorepos great?