Monorepo or Multirepo? Role-Based Repositories • Hacker News Board

Re: Monorepo or Multirepo? Role-Based Repositories

#91

Post by fragmede » Sun, Oct 13, 2019, 2:42 AM UTC

tom_ wrote: post #18

weberc2 wrote: post #12

Earlier quoted context omitted.

Can you elaborate on “monorepos do not prevent you from checking packages into source control” and how that helps to avoid recompiling everything? Why would you check a package into source control anyway? Surely source control is for source code? And I lean toward monorepos, btw, but there are still lots of obstacles and monorepo proponents don’t tend to acknowledge them or offer clear suggestions for how to solve or…

It's version control, not necessarily just source control! If something could benefit from being versioned, why would you not check it in? You then guarantee everybody has the same version. That's exactly what this thing is there for. Git's design can limit its usefulness in this respect - though perhaps you could solve this to some extent with git LFS? - but not all version control systems have this problem.

git annex (or git LFS, if you buy into github's NIH) is requisite if you want to use git like this, broadly. git will happily store any and all binaries you ask it to, but upon (blind) checkout, it will grab every single revision of said binary, taking up as much however much space that takes.

(partial clones avoid this, but, as git isn't designed for this use case, grabbing all of history happens far too easily.)

Re: Monorepo or Multirepo? Role-Based Repositories

#92

Post by fragmede » Sun, Oct 13, 2019, 3:03 AM UTC

username90 wrote: post #14

Cedricgc wrote: post #3

The main driver of success in either model is in the tooling and practices invested in it to make it work in an organization. Google is successful with their monorepo because they have invested in building (blaze), source control (piper, code search), and commit to always developing on HEAD. Multirepo is currently easier for most companies because most public tooling (git, package manager) is built around multirepos.…

I wonder why nobody have made a good public monorepo offering similar to what Google have internally. Would probably be a hit at many companies since it fixes so many issues related to working in very large teams.

Git VFS, to allow monorepos to be operable at enterprise scale (courtesy of Microsoft) is out there, as is Uber's white paper on ML to predict branch merge compilation success - an actual problem at enterprise scale. That doesn't cover all issues but every (huge) company's workflow is different, thus there's no "one size fits all" that's totally suitable - part of git's success is that it fits into whatever existing workflow a company already has. So I'm not sure the full suite of Google's tools would be at hit at all companies, especially anything smaller, and less dedicated, than. Google. Having a team to work on and another to operate, Bazle (and the internal Google project it springs from); not all enterprises are willing to staff that work, and definitely not as strongly as Google.

Re: Monorepo or Multirepo? Role-Based Repositories

#93

Post by tomrod » Sun, Oct 13, 2019, 3:04 AM UTC

gnachman wrote: post #71

Cedricgc wrote: post #3

The main driver of success in either model is in the tooling and practices invested in it to make it work in an organization. Google is successful with their monorepo because they have invested in building (blaze), source control (piper, code search), and commit to always developing on HEAD. Multirepo is currently easier for most companies because most public tooling (git, package manager) is built around multirepos.…

When I started at Google the tooling was not very good, and monorepo was pretty painful. They used perforce, and it simply couldn't keep up. Commits could take minutes. Code review was also unbearably slow. Blaze didn't exist yet; just before I started they had tools that generated million+ line makefiles that everyone hated. So yeah, you need good tooling, but even Google didn't have it for the first ~10 years of it…

I went to graduate school with a guy who ended up on Google's infrastructure team. He'd previously worked as head or lead dev for subversion (a little hazy on the details).

We worked on a small project where we put together statistical measures for codebases. It was a lot of fun, even if the infrastructure was out of my wheelhouse at the time.

Folks that can manage billion-line codebases are on a whole different level I think. I wonder sometimes how many folks like that there are.

EDIT: Looks like he left for a bit and is now back. Good on him!

Re: Monorepo or Multirepo? Role-Based Repositories

#94

Post by huherto » Sun, Oct 13, 2019, 3:07 AM UTC

givehimagun wrote: post #34

scarmig wrote: post #27

I'm curious: how would most people here define monorepo vs multirepo? On the surface, most people seem to think of a monorepo as a source control management system that exposes all source code as if it's a traditional filesystem accessed through a single point of entry. Multirepo, in contrast, seems to be about multiple points of entry. But that's a superficial and uninteresting distinction. All the hard parts of man…

I've shifted to focusing on repo == team. If your organizational structure is to have many little teams that are independent from each other, then you build your source code management to reflect that. I built my engineering staff to focus on any of the initiatives that my boss hands to me (changes week/week) - so we went monorepo so we could move between those projects/apps/programs quickly. We knew that we didn't w…

This is my conclusion too. Team becomes fundamental entity and projects/products belong to teams. Everything the team produces stays on the team's repo. So you always know which team owns what. It is also easier to review, supervise and clean up.

Re: Monorepo or Multirepo? Role-Based Repositories

#95

Post by dmoy » Sun, Oct 13, 2019, 3:23 AM UTC

djhaskin987 wrote: post #61

username90 wrote: post #25

Earlier quoted context omitted.

It is just that Googles tooling around code works really well together. Code search to view code and directory based history so you aren't swamped by others commits, tap to run all unit tests all the time but with sectioned projects so you don't run all tests on every presubmit, sponge to gather every test log ever (even for the tests you run locally) so you can link full test logs to coworkers when you have problems…

> directory based commits This whole thread is interesting because subversion is exactly this, and works with large code bases. We used to have these told and we moved away from them.

Subversion works with large code bases, but not crazy massive codebases.

The version control is just the tip of the iceberg though, and is largely a solved problem: git or subversion, then perforce or straight to Facebook's mercurial stuff.

It's the other tooling that breaks on large enough mono repos that you have hard time with publicly. Searching your code takes awhile. Cross references don't go wide enough or take too long to generate. Builds take too long. Refactoring tools either take too long or don't support the repo of sufficiently large size.

Mostly this isn't a problem though, because few repos are actually large enough to cause real problems.

Re: Monorepo or Multirepo? Role-Based Repositories

#96

Post by catalogia » Sun, Oct 13, 2019, 4:19 AM UTC

blandflakes wrote: post #90

catalogia wrote: post #66

Earlier quoted context omitted.

When I was there, they were migrating away from perforce because they could no longer scale perforce fast enough to meet demand. I've not seen this talked about much outside of Amazon. It was also a huge day-to-day quality of life improvement for the users (the developers.) There are UX problems with git, but they pale in comparison to the UX problems with perforce which is truly unpleasant software.

The Alexa division migrated aggressively to git as soon as it was available and nobody publicly voiced any regret about losing perforce.

Several of the people I worked with at Amazon were skeptical of git, at least initially. Some people prefer tools they already know, prefer the routine and habit over learning a new tool. And I totally respect that by the way, git's UX is superior in mainly aesthetic ways, in terms of tactical productivity it's more of a wash. I still think git has the edge, but there is nothing to say a seasoned developer who's used perforce for years isn't being exceptionally productive with it.

Nearly everybody I talked to about it eventually came around to prefer git though. Once you've been forced to swallow the bitter pill of learning something new and changing your workflow, I think the advantages begin to shine through.

On the other hand, maybe I'm just biased because I was proficient in git years before I was ever exposed to perforce. So maybe it was myself who was balking at learning something new, and that's why I was so relieved when my team switched to git. But I do genuinely believe that git has a superior UX.

Re: Monorepo or Multirepo? Role-Based Repositories

#97

Post by OneMoreGoogler » Sun, Oct 13, 2019, 4:21 AM UTC

Cedricgc wrote: post #3

The main driver of success in either model is in the tooling and practices invested in it to make it work in an organization. Google is successful with their monorepo because they have invested in building (blaze), source control (piper, code search), and commit to always developing on HEAD. Multirepo is currently easier for most companies because most public tooling (git, package manager) is built around multirepos.…

You know Google's monorepo is successful because they have so many of them! google3, Android, Chrome browser, ChromeOS...

Kidding aside, my point is Google recognizes obvious boundaries between e.g. their web stuff and android, and organizes their code accordingly.

Re: Monorepo or Multirepo? Role-Based Repositories

#98

Post by Skunkleton » Sun, Oct 13, 2019, 5:45 AM UTC

likeliv wrote: post #52

avisser wrote: post #32

Earlier quoted context omitted.

Even with submodules it's still a PR per repo. Global, atomic changes are super powerful.

There is another tool called git-subtree that should solve these problems, I think. But I've never seen it in use

git subtree is just a wrapper around subtree merges, I don't think they solve the same problem as submodules.

Re: Monorepo or Multirepo? Role-Based Repositories

#99

Post by likeliv » Sun, Oct 13, 2019, 7:21 AM UTC

nhumrich wrote: post #73

likeliv wrote: post #47

Earlier quoted context omitted.

Calling eachother at network level is still a dependency. (And even a build dependency if you use something like protobuff or other protocol description files)

A network dependency is not a build dependency. Protobuf files should be copy pasted, not referenced directly. Saying you need a single repo to build correctly for your network dependencies is like saying you cant use a third party system (aws, etc.) Without having a link to their code base.

Point is that when you want to do a change in the "API" (or call it "protocol"), you need to touch the different repositories and coordinate to use the right versions together.

About the copy/paste of protobuf files, it works but makes it more difficult to keep them in sync.

And I did not say you need a single repo. I'm saying the stated disadvantages of multi repo are real.

Re: Monorepo or Multirepo? Role-Based Repositories

#100

Post by Too » Sun, Oct 13, 2019, 7:23 AM UTC

givehimagun wrote: post #34

scarmig wrote: post #27

I'm curious: how would most people here define monorepo vs multirepo? On the surface, most people seem to think of a monorepo as a source control management system that exposes all source code as if it's a traditional filesystem accessed through a single point of entry. Multirepo, in contrast, seems to be about multiple points of entry. But that's a superficial and uninteresting distinction. All the hard parts of man…

I've shifted to focusing on repo == team. If your organizational structure is to have many little teams that are independent from each other, then you build your source code management to reflect that. I built my engineering staff to focus on any of the initiatives that my boss hands to me (changes week/week) - so we went monorepo so we could move between those projects/apps/programs quickly. We knew that we didn't w…

Sounds like that would enforce Conway's Law a lot and make cross team collaboration more difficult.