Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Ingredients of a Productive Monorepo

blog.swgillespie.me

Post reply on HN
Print view View original

71–80 of 268 posts

Re: The Ingredients of a Productive Monorepo

#71

Post by rwieruch »

Over the past four years, I’ve set up three monorepos for different companies as contract work. The experience was positive, but it’s essential to know your tools.

Since our monorepos were used exclusively for frontend applications, we could rely entirely on the JavaScript/TypeScript ecosystem, which kept things manageable.

What I learned is that a good monorepo often behaves like a “polyrepo in disguise.” Each project within it can be developed, hosted, and even deployed independently, yet they all coexist in the same codebase. The key benefit: all projects can share code (like UI components) to ensure a consistent look and feel across the entire product suite.

If you're looking for a more practical guide, check out [0].

[0] https://www.robinwieruch.de/javascript-monorepos/

Re: The Ingredients of a Productive Monorepo

#72

Post by zelphirkalt »

dezgeg wrote: post #39
friendzis wrote: post #23

Earlier quoted context omitted.

Exactly. Take your monorepo, split it into n repos by directory at certain depth from root, write very a rudimentary VCS wrapper script to sync all the repos in tandem and you have already solved a lot of pain points. > E.g. it wouldn't be too hard to create "linked" PRs across repos for changes that span projects, with linked testing infrastructure Bitbucket does this out-of-the box :)

> Take your monorepo, split it into n repos by directory at certain depth from root, write very a rudimentary VCS wrapper script to sync all the repos in tandem and you have already solved a lot of pain points. Then you lose the capability to atomically make a commit that crosses repoes. I'm not sure if there is any forge that allows that, except Gerrit might with its topics feature (I've not gotten the opportunity t…

You could also use git submodules in an overarching separate repo, if you want to lock down a set of versions. It doesn't even have to affect the submodule repos in any way. That would simplify branches in the single repos and enable teams to work independently on each repo. Then you only deploy from the overarching repo's main branch for example, where you have to create PRs for merging into the main branch and get it reviewed and approved.

Re: The Ingredients of a Productive Monorepo

#73

Post by bittermandel »

I firmly believe that us at Molnett(serverless cloud) going for a strict monorepo built with Bazel has been paramount to us being able to make the platform with a small team of ~1.5 full-time engineers.

We can start the entire platform, Kubernetes operators and all, locally on our laptops using Tilt + Bazel + Kind. This works on both Mac and Linux. This means we can validate essentially all functionality, even our Bottlerocket-based OS with Firecracker, locally without requiring a personal development cluster or such.

We have made this tool layer which means if I run `go` or `kubectl` while in our repo, it's built and provided by Bazel itself. This means that all of us are always on the same version of tools, and we never have to maintain local installations.

It's been a HUGE blessing. It has taken some effort, will take continuous effort and to be fair it has been crucial to have an ex Google SRE on the team. I would never want to work in another way in the future.

EDIT: To clarify, our repo is essentially only Golang, Bash and Rust.

Re: The Ingredients of a Productive Monorepo

#74

Post by zelphirkalt »

nssnsjsjsjs wrote: post #53
echelon wrote: post #12

Earlier quoted context omitted.

As a former IC at a large monorepo company, I preferred monorepos over polyrepos. It was the "THE" monorepo, and it made understanding the company's service graph, call graph, ownership graph, etc etc. incredibly clear. Crystal clear. Vividly so. Polyrepos are tribal knowledge. You don't know where anything lives and you can't look or discover it. Every team does their own thing. Inheriting new code is a curse. Code…

Monorepo monoliths make it hard to experiment. Getting something as benine as a later version of .NET becomes a mammoth task requiring the architecture team and everything stays old. Want to use a reasonable tool? No chance.

I don't see how it immediately has to follow from monorepo usage, that its parts cannot have separate runtimes and dependency versions. Perhaps the monorepo tooling is still that bad, idk, but there seems no inherent reason for that.

Re: The Ingredients of a Productive Monorepo

#75

Post by oivey »

CamouflagedKiwi wrote: post #45
AlotOfReading wrote: post #14

One thing I don't usually see discussed in monorepo vs multi repo discussions is there's an inverse Conway's law that happens: choosing one or the other will affect the structure of your organization and the way it solves problems. Monorepos tend to invite individual heroics among common infrastructure teams, for example. Because there are so many changes going in at once, anything touching a common area has a huge n…

That's an optimistic take on what happens in the polyrepo setup. A common alternative (I suspect by far the more common one) is that changes are made to the common area but not propagated to downstream repos, which all end up pinned to different versions of the common repo and struggle to update once they get ~years out of date.

Yeah. My experience is that the teams managing shared repos tend to shift responsibility for integrating their changes onto their users. They then also more often make breaking changes because they’re insulated from the costs of those changes.

Re: The Ingredients of a Productive Monorepo

#76

Post by chrismatic »

The point about trying to stick with a single language build tooling really cannot be stressed enough. It is what prompted me to write a simplified version of Bazel, a generic "target determinator" with caching capabilities if you will. I call it "Grog", the monorepo build tool for the grug-brained developer.

https://grog.build/why-grog/

Re: The Ingredients of a Productive Monorepo

#77

Post by Adverblessly »

AlotOfReading wrote: post #14

One thing I don't usually see discussed in monorepo vs multi repo discussions is there's an inverse Conway's law that happens: choosing one or the other will affect the structure of your organization and the way it solves problems. Monorepos tend to invite individual heroics among common infrastructure teams, for example. Because there are so many changes going in at once, anything touching a common area has a huge n…

> Because there are so many changes going in at once, anything touching a common area has a huge number of potential breakages, so effort to deliver even a single "feature" skyrockets.

If a specific change in a monorepo is so centrally embedded it requires incredible effort to do atomically (the benefit of having the monorepo in the first place), you are still able to split it into multiple gradual changes (and "require coordinating several PRs over a couple of weeks and some internal politics, but that might also be split among different developers who aren't even on a dedicated build team.").

So in a monorepo you can still enjoy the same advantage you describe for multi repo, and you'll even have much better visibility into the rollout of your gradual change thanks to the monorepo.

Re: The Ingredients of a Productive Monorepo

#78

Post by oivey »

friendzis wrote: post #23
shawabawa3 wrote: post #19

Earlier quoted context omitted.

Every time I've been monorepos compares with polyrepos it's always "monorepo plus millions of dollars of custom tool engineering" vs "stock polyrepo" Why can't we add millions of dollars of tool engineering on top of polyrepos to get some of the benefits of monorepos without a lot of the pain? E.g. it wouldn't be too hard to create "linked" PRs across repos for changes that span projects, with linked testing infrastr…

Exactly. Take your monorepo, split it into n repos by directory at certain depth from root, write very a rudimentary VCS wrapper script to sync all the repos in tandem and you have already solved a lot of pain points. > E.g. it wouldn't be too hard to create "linked" PRs across repos for changes that span projects, with linked testing infrastructure Bitbucket does this out-of-the box :)

That sort of directory-based splitting almost never works in my experience. The code between those directories is almost always tightly coupled. Splitting arbitrarily like this gives the illusion of a non-tightly coupled code base with all the disadvantages of highly coupled dependencies. It’s basically the worst possible way to migrate workflows.

Re: The Ingredients of a Productive Monorepo

#79

Post by boxed »

The article links to a site with this definition:

> A monorepo is a single repository containing multiple distinct projects, with well-defined relationships.

It would be better if there were terms that delineated "one repo for the company" from "one repo per project" from "many repos for a single project".

Re: The Ingredients of a Productive Monorepo

#80

Post by DanielHB »

lxe wrote: post #8

So there are 2 kinds of big tech monorepos. One is the kind described in the article here: "THE" monorepo of the (mostly) entire codebase, requiring custom VCS, custom CI, and a team of 200 engineering supporting this whole thing. Uber and Meta and I guess Google do it this way now. It takes years of pain to reach to this point. It usually starts with the other kind of "monorepo": The other kind is the "multirepo mon…

My company has been moving towards having monorepos per language stack. Decent compromise
Post reply on HN