Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Monorepo or Multirepo? Role-Based Repositories

blog.7mind.io

Post reply on HN
Print view View original

41–50 of 120 posts

Re: Monorepo or Multirepo? Role-Based Repositories

#41

Post by pshirshov »

01100011 wrote: post #6

Monorepo shortcomings 1 and 2 seem like bullshit to me. Perforce, the popular monorepo at most companies I've worked at, supports access control. Monorepos do not prevent you from segmenting your code into modules and pushing binary/source packages into source control so that builds can avoid compiling everything(TiVo used to do this, and it worked well when you got the hang of it). I feel like these debates are ofte…

> Monorepo shortcomings 1 and 2 seem like bullshit to me.

It's a blogpost and the author didn't try to build a total and exhaustive formal system. These shortcomings are not absolute truth but actually they are true.

I've seen this multiple times: a small projects evolves over years into a monster. Engineers add new components and reuse any other components they may need creating horizontal links. At some point they feel like they lost their productivity and they blame monorepo because it's easy to create horizontal links in a typical monorepo. So, they try to build a multirepo flow and they spend a lot of effort, time and money trying to make it working. At some point they feel that their productivity is even worse than it was before because now they need to orchestrate things so they merge everything back.

Same applies not only to VCS flows, but to system design as well.

When we discuss monolith/microservices controversy all the monorepo/multirepo arguments may be isomorphically translated to that domain. What is better, monolithic app or a bunch of microservices? A role-based app of course: https://github.com/7mind/slides/blob/master/02-roles/target/...

Re: Monorepo or Multirepo? Role-Based Repositories

#42

Post by jayd16 »

avisser wrote: post #32
jayd16 wrote: post #29

Whats the current state of git submodules? It seems like you could get some of the benefits of mono-repos in that you can reference dependency projects directly like a mono-repo. You can, in theory, treat many projects like a single code base. I don't see it used very often though. Why not?

Even with submodules it's still a PR per repo. Global, atomic changes are super powerful.

Good point! Hmm. Maybe git(hu|la)b could solve this with some kind of pull request batching.

Re: Monorepo or Multirepo? Role-Based Repositories

#43

Post by jayd16 »

weberc2 wrote: post #39
jayd16 wrote: post #17

Earlier quoted context omitted.

You can use something like a shared binary repo such as maven or you could just check in dependencies and not worry about an external server being available for builds. >Surely source control is for source code? This is just pedantry. Checking in binaries is a pragmatic solution that solves a lot of problems.

I wasn’t trying to be a pedant, I’ve just never heard of anyone doing this. I was wondering how it helped solve the problem of not rebuilding everything.

In short, the binaries are already built. Usually its faster to link to a prebuilt binary than to build from scratch.

Re: Monorepo or Multirepo? Role-Based Repositories

#44

Post by 01100011 »

jsnell wrote: post #24
01100011 wrote: post #6

Monorepo shortcomings 1 and 2 seem like bullshit to me. Perforce, the popular monorepo at most companies I've worked at, supports access control. Monorepos do not prevent you from segmenting your code into modules and pushing binary/source packages into source control so that builds can avoid compiling everything(TiVo used to do this, and it worked well when you got the hang of it). I feel like these debates are ofte…

VCS access control are the wrong tool for solving the "people use code they shouldn't" complaint. First, VCS ACLs will massively reduce the benefits you're supposed to get from a monorepo. How will you do global refactors in that kind of a situation? How does a maintainer of a library figure out how the clients are actually using it? (The clients must have visibility into the library, but the opposite it unlikely to…

VCS ACLs can allow for read-only access. You can also split public interfaces into their own header. If you want the maintainer of a library to be able to refactor clients of the library, then you have to grant them access to the client code. How does a multirepo solve this issue?

> How will VCS ACLs allow me to hide the implementation but not the interface?

If you don't give people access to the code, they can't build it. So what? Publish pre-built binaries from your CI system back to source control.

> At that point you don't have a monorepo, you've got multirepos stored in a monorepo.

I think it's a spectrum. It would be stupid to dogmatically stick to either extreme. You modify things in a pragmatic fashion to solve the problems you're facing. In my experience, starting with a monorepo and making exceptions as needed has worked better than the alternative.

Your post sounds similar to a lot of the multi/mono repo discussions. You've focused on one problem and one way to solve that problem without considering that there are many ways to work around it. Neither approach is going to be pain-free and both require tooling for special scenarios.

Re: Monorepo or Multirepo? Role-Based Repositories

#45

Post by 01100011 »

username90 wrote: post #23
weberc2 wrote: post #12

Earlier quoted context omitted.

Can you elaborate on “monorepos do not prevent you from checking packages into source control” and how that helps to avoid recompiling everything? Why would you check a package into source control anyway? Surely source control is for source code? And I lean toward monorepos, btw, but there are still lots of obstacles and monorepo proponents don’t tend to acknowledge them or offer clear suggestions for how to solve or…

At Google we check in the source of every library into the monorepo and compile them ourselves with cached builds from a central server, I don't think we use package managers.

You don't have to use a package manager, that's just the approach the TiVo folks came up with a couple decades ago. They use RPM to package independent software modules and check them into (IIRC) a separate build repository which saves the last n months of work. A local config file is used to choose the binary package version to use, or, alternatively, the locally built files to use. They probably could have just made tarballs, since I don't think they used any of the dependency checking.

Re: Monorepo or Multirepo? Role-Based Repositories

#47

Post by likeliv »

nhumrich wrote: post #46

The cons to multi repo are all anti patterns for microservices anyway. If you're doing microservices you shouldn't have build dependencies on other projects. The should only call eachother at a network level.

Calling eachother at network level is still a dependency. (And even a build dependency if you use something like protobuff or other protocol description files)

Re: Monorepo or Multirepo? Role-Based Repositories

#48

Post by pshirshov »

nhumrich wrote: post #46

The cons to multi repo are all anti patterns for microservices anyway. If you're doing microservices you shouldn't have build dependencies on other projects. The should only call eachother at a network level.

But... we do better things than microservices: https://github.com/7mind/slides/blob/master/02-roles/target/...

Re: Monorepo or Multirepo? Role-Based Repositories

#49

Post by weberc2 »

jayd16 wrote: post #43
weberc2 wrote: post #39

Earlier quoted context omitted.

I wasn’t trying to be a pedant, I’ve just never heard of anyone doing this. I was wondering how it helped solve the problem of not rebuilding everything.

In short, the binaries are already built. Usually its faster to link to a prebuilt binary than to build from scratch.

So where do these binaries get built and how does the system know which binaries to rebuild for a given change? If developers are building binaries and committing them directly, doesn’t that open up security or even correctness issues? How does this approach satisfy compliance concerns (how can the CTO or a manager sign off on the changes that went into the binary if it’s just something a random developer committed?)? How does this scale to tens of deployments per day? These are hard monorepo problems, and they keep being handwaved away.

Re: Monorepo or Multirepo? Role-Based Repositories

#50

Post by megous »

pricechild wrote: post #33
megous wrote: post #26

Earlier quoted context omitted.

I think that whether to use mono/multi repo depends on whether you're willing to dump money into updating everyting at once, or not. If not, monorepos are really a big hindrance. It's better to split on the project boundary (things that may have different development paces), and use git worktree for having different versions of libraries checked out for building/bundling. It works fairly nicely with meson, as you can…

> whether you're willing to dump money into updating everyting at once I think the majority of projects in this world only update everything at once. They haven't investing in testing, sensible api's and testing to allow updating small pieces of their solution. From my experience, I also think the majority of people who think they have a library and need multi repos to deal with that, don't have a library. To further…

Project != company. Project != consultancy.

Monorepo just didn't work for me. I have ~10 web projects for different customers + my personal projects that use various versions of some common code.

It doesn't make any financial sense to evolve my common code by updating all the customer's code for free when they're not even asking for it. So on this level it doesn't work.

Even within a single company with just two devs and around 90 repos for the main product and plugins, it was hard to justify making a mono-repo with plain git, because plugins and the main app had different release schedules, priorities, so it never really happened that it was economical to port all plugins right away to the new version of the main app on every change.

I still think going multi vs mono is a business decision, rather than a technical one. You'll have to have special tooling for either case, just a different one.

Post reply on HN