Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google stores billions of lines of code in a single repository (2016) [pdf]

dl.acm.org

Post reply on HN
Print view View original

141–150 of 209 posts

Re: Google stores billions of lines of code in a single repository (2016) [pdf]

#141

Post by 8note »

yongjik wrote: post #84
spion wrote: post #51

Earlier quoted context omitted.

> any breaking changes should be dealt with in the same change as the version bump Does this mean that some things will never get updated, as the effort required is impossibly high?

Effort to update something is high because there's a lot of code, not because it's in a monorepo. Updating the same code scattered across multiple repositories takes as much work in the best case. More realistically, some copy of the same code will stay unupdated because the cost to track down every repository in the company is too much.

Lots of code under the assumption that all the code needs to use the same version*

A big bang always sucks versus some migration over time

Re: Google stores billions of lines of code in a single repository (2016) [pdf]

#142

Post by mook »

ywoski wrote: post #126
spion wrote: post #51

Earlier quoted context omitted.

> any breaking changes should be dealt with in the same change as the version bump Does this mean that some things will never get updated, as the effort required is impossibly high?

Almost. We had a UI library on Android that was stuck on an alpha version of the library for three or so years after the library had shipped. Upgrading the library broke many tests across the org, and no one wanted to own going in and getting each team to fix it. Eventually, the library had a v2 release, and people started to care about being able to use it. Ultimately, they just forked the current release and append…

> Ultimately, they just forked the current release and appended a v2 to the package name.

Hmm, does that explain the golang module versioning requirement where v2 must have a different name?

Re: Google stores billions of lines of code in a single repository (2016) [pdf]

#143

Post by 8note »

ywoski wrote: post #131
coryrc wrote: post #130

Earlier quoted context omitted.

> The monorepo works for Google Does it? Or is it stopping Google from supporting products which only make millions in revenue because of the massive burden of continually updating?

Oh geez, that's an entirely different can of worms that isn't related to the monorepo. Most products at Google are not dropped because the monorepo makes it difficult for them to support - and I'm not sure how it would or how you got to that association. Also, plenty of products that are killed are not in the monorepo. They are usually dropped due to a mix of things, but a big part is just better product management.

Better project management as in, somebody politicked their way into owning a replacement for a currently running thing?

The implemented product, as well as the vision for something like inbox or Google music is still way better than Gmail and YouTube music as the end user

Re: Google stores billions of lines of code in a single repository (2016) [pdf]

#144

Post by hota_mazi »

lopkeny12ko wrote: post #35

There's a lot of love for monorepos nowadays, but after more than a decade of writing software, I still strongly believe it is an antipattern. 1. The single version dependencies are asinine. We are migrating to a monorepo at work, and someone bumped the version of an open source JS package that introduced a regression. The next deploy took our service down. Monorepos mean loss of isolation of dependencies between ser…

> The single version dependencies are asinine. We are migrating to a monorepo at work, and someone bumped the version of an open source JS package that introduced a regression. The next deploy took our service down

You're doing it wrong.

The point of monorepo is that if someone breaks something, it breaks right away, at build time, not at deployment time.

You're not really using a monorepo.

Re: Google stores billions of lines of code in a single repository (2016) [pdf]

#145

Post by 8note »

admax88qqq wrote: post #77
spankalee wrote: post #63

Earlier quoted context omitted.

Even so, the cost is often outrageously high. No to mention that if you're the first team to import a third_party library, you own it and other teams can add arbitrary cost to you updating it. You have to be very aggressive with visibility and SLAs to work around this.

You're basically just describing all the pain with pulling in a dependency regardless of monorepo or not. If the third party dependency does not add enough value to justify the cost then don't add it.

With multirepo setups, you don't necessarily need to update the package for all code at all.

Instead, some newer package completely replaces an old one, with no relation to the old dependency package, or with a dependency on some future one, and both can run at the same time while turning the old one off

Re: Google stores billions of lines of code in a single repository (2016) [pdf]

#146

Post by forgotusername6 »

ameliaquining wrote: post #32
forgotusername6 wrote: post #29

Earlier quoted context omitted.

I imagine looking for vulnerable areas of the code might be something people would be interested in doing. Maybe start with login or billing or something. You could also look at recent activity to spot new, unannounced projects. You could use blame to find who wrote what and target them for anything from job offers to social engineering attacks.

Most of that information is readily available on the corporate intranet without having to dig through source code. Security-by-obscurity isn't something to rely on (again, except in the case of things like abuse detection where there's no alternative).

You aren't necessarily looking for things that would be defined as security by obscurity. You're looking for bugs with a security implication. With the source code, you can look for these bugs without arousing suspicion.

Re: Google stores billions of lines of code in a single repository (2016) [pdf]

#147

Post by 8note »

tho2i3u4o3i434 wrote: post #133
spion wrote: post #51

Earlier quoted context omitted.

> any breaking changes should be dealt with in the same change as the version bump Does this mean that some things will never get updated, as the effort required is impossibly high?

Google's software mostly uses dependencies already in the google monorepo, so these issues don't crop up. The person/team working on library changes have to ensure that nothing breaks, or the downstream users are notified early on. Don't think this would apply to many companies.

That sounds like a huge amount of effort unrelated to your current project, both for those being forced to upgrade, and those organizing the upgrade

Re: Google stores billions of lines of code in a single repository (2016) [pdf]

#148

Post by I_AM_A_SMURF »

taeric wrote: post #120
jcranmer wrote: post #119

Earlier quoted context omitted.

> In a multi-repo, you don't build source dependencies between projects. In my experience with software projects, this is very much not the case. It's one of the main reasons I'm such a big fan of monorepos--I have been burned way too many times by the need to make atomic commits involving separate repositories.

If you have multiple repos, you can't have an atomic commit between them. Pretty much period. I'm scared to hear what you mean on that. Ideally, all tooling makes the separate nature of the projects transparent. They should test separately. They should deploy separately. If that is not the case, then yes, they should be in the same repo.

I've worked at places where they would "solve" this problem by letting the build break while all the commit in various repositories land all at once. It's really bad.

Re: Google stores billions of lines of code in a single repository (2016) [pdf]

#149

Post by 8note »

throwaway2037 wrote: post #95
jmillikin wrote: post #46

Earlier quoted context omitted.

> The single version dependencies are asinine. We are migrating to > a monorepo at work, and someone bumped the version of an open > source JS package that introduced a regression. There's no requirement to have single versions of dependencies in a monorepo. Google allows[0] multiple versions of third-party dependencies such as jQuery or MySQL, and internal code is expected to specify which version it depends on. > I…

I never worked at Google, but this post sums up everything I had to say about the matter. GP has a sh-tty monorepo experience at one company and decides to make a statement about another company where they never worked (so I presume). HN absurdism as its best! I second your point about monorepo versus ball of mud. They are so different. And managing all of this is about social/culture, less science-y. If you don't ha…

It generally gives the sense that mono-repo is actually irrelevant, and the more detailed processes across the whole experience are what matters.

Re: Google stores billions of lines of code in a single repository (2016) [pdf]

#150

Post by gorgoiler »

Imagine you have two teams in one monorepo and requirements.txt has pinned numpy at 1.22. One team wants to upgrade to 1.24 but the upgrade breaks the other team’s code as it was dependent on an emergent property* in the older version of numpy.

How would you handle this situation as an IC? As a manager of one of the teams? As a skip-level manager of both teams?

As a budding IC on the team that wants the upgrade, you may want to go fix up the other team’s code for them so you can bring them along with the upgrade. Realistically, the further you get from Google’s level of engineering discipline and skill the more likely you are to encounter the following in the needs-1.22 codebase:

- horrible code that is hard to understand and therefore hard to refactor

- code with no tests, making it risky to refactor

- the team that wrote it have all left or been fired and no one is available to help understand it

- they are a remote team with no social relationship to you who interact entirely online, in writing, in the style of an aggressive subreddit mod

- deeply entrenched factions mean that even if you offer them a patch they will default refuse it because who are you to work on their codebase and they don’t need the upgraded numpy so why should they waste resources on reviewing something they don’t want

- misguided adherence to status enhancing terms like “audit” and “compliance” mean jobsworth ICs refuse to even look at your patch because someone somewhere once heard a friend of a friend whose company failed SOC2 because engineer from floor X made a change to code owned by floor Y and it went against policy

All of these social problems are real ones I have encountered and if you have solved these then you’re probably already happily in a monorepo already. If instead you work in an org full of teams pointing guns at each other in a fight to the death to stop any kind of cross org collaboration from sullying the purity of the tribal system then know this: it gets better, and if you build the right social connections then the technical efficiency of having your monobusiness executing its monomission inside a monorepo is within reach!

*bug

Post reply on HN