I worked in a big bank in the UK using monorepo "cuz Google uses it", error number 1, your not Google.
The clones were gigantic, Jenkins would timeout cloning the whole project when all it needed was a bunch of files.
Merge conflicts all over the place, but the best part, we had scripts on our pipeline literally removing folders after cloning the repo to avoid automatic inclusions of libs etc.
In my opinion separation of boundaries is one of those things that should t be mess with.
Monorepo or Multirepo? Role-Based Repositories
111–120 of 120 posts
Re: Monorepo or Multirepo? Role-Based Repositories
#112GoCD provides “fan in” which supports monorepos
Re: Monorepo or Multirepo? Role-Based Repositories
#113I worked in a big bank in the UK using monorepo "cuz Google uses it", error number 1, your not Google. The clones were gigantic, Jenkins would timeout cloning the whole project when all it needed was a bunch of files. Merge conflicts all over the place, but the best part, we had scripts on our pipeline literally removing folders after cloning the repo to avoid automatic inclusions of libs etc. In my opinion separatio…
Monorepos with Git don't play together nicely. Perforce is key if you have lots of devs on a monorepo.
Re: Monorepo or Multirepo? Role-Based Repositories
#114Earlier quoted context omitted.
In short, the binaries are already built. Usually its faster to link to a prebuilt binary than to build from scratch.
So where do these binaries get built and how does the system know which binaries to rebuild for a given change? If developers are building binaries and committing them directly, doesn’t that open up security or even correctness issues? How does this approach satisfy compliance concerns (how can the CTO or a manager sign off on the changes that went into the binary if it’s just something a random developer committed?)…
> So where do these binaries get built and how does the system know which binaries to rebuild for a given change?
By the CI. All major CI/CD tools support rules like build binary x whenever a file under x-src/* changes; commit binary x when the ref matches /v[0-9.]+/; don't allow developers to manually push to these refs / paths; (run a script to) bump the dependent x of y whenever binary x changes; merge the bumped version if all tests still pass; etc.
Re: Monorepo or Multirepo? Role-Based Repositories
#115Earlier quoted context omitted.
Google uses no versioned libraries?
Libraries internal to Google are kept at the latest and consumers are updated to use the latest APIs. 3rd party libraries are checking into the monorepo at a specific version and everyone uses the same version.
It's documented here
Re: Monorepo or Multirepo? Role-Based Repositories
#116Earlier quoted context omitted.
Libraries internal to Google are kept at the latest and consumers are updated to use the latest APIs. 3rd party libraries are checking into the monorepo at a specific version and everyone uses the same version.
It's documented here https://opensource.google/docs/thirdparty/
[deleted]
Re: Monorepo or Multirepo? Role-Based Repositories
#117Earlier quoted context omitted.
So where do these binaries get built and how does the system know which binaries to rebuild for a given change? If developers are building binaries and committing them directly, doesn’t that open up security or even correctness issues? How does this approach satisfy compliance concerns (how can the CTO or a manager sign off on the changes that went into the binary if it’s just something a random developer committed?)…
> So where do these binaries get built and how does the system know which binaries to rebuild for a given change? By the CI. All major CI/CD tools support rules like build binary x whenever a file under x-src/* changes; commit binary x when the ref matches /v[0-9.]+/; don't allow developers to manually push to these refs / paths; (run a script to) bump the dependent x of y whenever binary x changes; merge the bumped…
The problem is dependency graphs aren’t strictly hierarchical, so it doesn’t suffice to say “rebuild whenever something under this directory changes”.
Re: Monorepo or Multirepo? Role-Based Repositories
#118Earlier quoted context omitted.
(Sorry for being terse—on mobile). Validate the binary matches what? If the compiler has to compile the artifact to verify the artifact provided by the developer, why bother having the developer commit the artifact? The CI system could just do it. Never mind that having a bit-for-bit reproducible build is incredibly difficult. Anyway, such simple cases where a whole app lives under a single directory are vanishingly…
>The CI system could just do it. Depends if you want to wait for the CI system to upload or not. Also if you want CI to have commit permissions. >Never mind that having a bit-for-bit reproducible build is incredibly difficult. Debian is at something like 90% reproducible packages once they fix two outstanding things. Most languages will have settings and best practices at this point that will give reproducible builds…
> Depends if you want to wait for the CI system to upload or not. Also if you want CI to have commit permissions.
I guess you could deploy first and verify automatically later. Hadn’t thought of that.
> Debian is at something like 90% reproducible packages once they fix two outstanding things. Most languages will have settings and best practices at this point that will give reproducible builds.
Never the less, getting (and keeping) bit-for-bit reproducibility is a ton of work, especially for software that changes every day, and the benefits aren’t compelling for many projects.
> Then use Bezel once you get past that stage.
This seems to be the answer, but it’s not very satisfying since Bazel’s support for many popular languages (e.g., Python) is lacking and there are lots of rough edges to iron out.
> Look, to be blunt, it seems like you're trying to nitpick whatever anyone says while ignoring large parts of answers. Fact is, many people at small and large companies use monorepos successfully. They work for those people, you can keep trying to argue they don't or try to learn why they do.
I never understand why people get defensive about things like this. I’m not attacking monorepos. I manage a monorepo at my small company, and I’ve run into lots of issues trying to make it work. I’m here trying to understand why so many people rave about monorepos, but often don’t have good answers for things like “how to manage rebuilds?”. You see this as “nitpicking”, but the distinction between “just git diff a directory!” and “use something like Bazel” is important.
Re: Monorepo or Multirepo? Role-Based Repositories
#119I bet you could address this with a third approach: metarepo. The metarepo is a repo that uses sub modules to combine your multi repo ecosystem into a simulated monorepo. The metarepo is what ultimately gets built and deployed—no versioned dependencies to manage. Local development usually happens at the multirepo level, and the metarepo is managed mostly via CI.
Good idea. But what gets checked in at the metarepo level? The names of the branches that are checked out in the submodules under it? Can you have two metarepos, each with its own set of checked-out branches of the same original submodules?
I believe submodules track specific branches.