Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Ingredients of a Productive Monorepo

blog.swgillespie.me

Post reply on HN
Print view View original

211–220 of 268 posts

Re: The Ingredients of a Productive Monorepo

#211

Post by yencabulator »

crazygringo wrote: post #209
yencabulator wrote: post #191

Earlier quoted context omitted.

> To me the whole point of a monorepo is atomic commits for the whole org. The belief that a monorepo makes a change somehow more atomic is one of the traps. From the article: > The greatest power and biggest lie of the monorepo is that it is possible to make atomic commits across your entire codebase. [...] > Your monorepo now contains many different deployable artifacts that deploy at different times. It is also te…

That's confusing two different things, though. A monorepo does make changes atomic in the code. There's no trap there. You're talking about deployment, and yes when deployment is staggered, then obviously all atomic changes need to be backward-compatible, or else be very carefully orchestrated. But that doesn't have anything to do with monorepo vs polyrepo. That's just staggered deployment. You have to deal with back…

There's no such thing as when deployment is staggered. It's a distributed problem, so by definition it is not synchronous.

(Or you turn services off for the duration of the deploy. Most companies do not want that these days.)

Also, you're missing this part of the article:

> While this is also possible in a world with many repositories, the requirement to do this change in multiple pull requests is often enough to remind engineers that breaking changes to a service contract are not safe to make.

Re: The Ingredients of a Productive Monorepo

#212

Post by layer8 »

datadrivenangel wrote: post #206
jyounker wrote: post #188

Earlier quoted context omitted.

Monolith != Monorepo. They're independent concepts.

The tooling defaults around Github encourage having one thing per repo.

A monolith can (and often does) consist of separately versioned libraries.

Re: The Ingredients of a Productive Monorepo

#213

Post by layer8 »

ecoffey wrote: post #171
cmrdporcupine wrote: post #103

Earlier quoted context omitted.

The push to fragmentation and atomism is so strong with this generation of devs. The obsession with microservices, dozens of small repositories, splitting everything up from fear of "monoliths." What they're doing is creating a mass of complexity that is turning org-chart problems into future technical ones and at the same time not recognizing the intrinsic internal dependencies of the software systems they're buildi…

In my experience microservices are easier to manage and understand when organized in a monorepo.

That indicates a strong coupling between those microservices.

Re: The Ingredients of a Productive Monorepo

#214

Post by WorldMaker »

bigbuppo wrote: post #159

It's kind of weird that both Microsoft and Google were both using Perforce. What does Perforce do that worked well at those companies for so long, and what caused them to dump it? Did they just get tired of the licensing cost? I think what I'm getting at is that maybe the real missing feature isn't whatever it is that allows you to make stupidly large monorepos, but that maybe we should add Perforce's client workspac…

Perforce didn't do anything extraordinarily well, it was just dumb enough it didn't do anything particularly poorly.

Perforce had a classic file locking model where a central server was in charge of file locks and a file was read-only until it was unlocked and the number of users that could unlock a file at the same time was often as low as 1.

So even if most Perforce operations were O(n^2) or worse, they were often only n = unlocked files, not n = files in repo. git status checks the full worktree, so is n = files in (visible part of) repo.

The "file is locked by another user" problem led to doing a lot of work outside Perforce itself. Often diff and patch tools and patch queues/changeset queue tools would proliferate around Perforce repos not provided by Perforce itself, but mini-VCSes built on top of Perforce. (Which is part of why Microsoft entirely forked Perforce early on. If you are already building a VCS toolkit on top of the VCS, might as well control that, too.)

A big point about git and its support for offline work, is that it works nothing like Perforce and you mostly don't want it to. A big benefit to git's model is that we mostly aren't using git as a low-level VCS toolkit and using a diaspora of other tools on top of git. (Ironically so, given git's original intent was to be the low-level VCS toolkit and early devs expected more "porcelain" tools to be built on top of it as third-party projects.)

Re: The Ingredients of a Productive Monorepo

#215

Post by layer8 »

przmk wrote: post #82
lxe wrote: post #8

So there are 2 kinds of big tech monorepos. One is the kind described in the article here: "THE" monorepo of the (mostly) entire codebase, requiring custom VCS, custom CI, and a team of 200 engineering supporting this whole thing. Uber and Meta and I guess Google do it this way now. It takes years of pain to reach to this point. It usually starts with the other kind of "monorepo": The other kind is the "multirepo mon…

At my current $dayjob, there is a backend that is split into ~11 git repos which results in a single feature being split among 4-5 merge requests and it's very annoying. We're about to begin evaluating monorepos to group them all (among other projects). What would the alternative to a monorepo be in this case, knowing that we can't bundle the repos together?

The general rule is that things should be versioned together that change together. Separate repositories should be thought of similarly to separately versioned libraries. Dependencies between repositories should have stable interfaces. Design decisions that are likely to change should be encapsulated within a module, so that these decisions are hidden from other modules (a seminal paper about that is [0]). These considerations should guide any split into separate repositories.

[0] https://wstomv.win.tue.nl/edu/2ip30/references/criteria_for_...

Re: The Ingredients of a Productive Monorepo

#216

Post by cloogshicer »

krschultz wrote: post #162
cloogshicer wrote: post #47

Here's what I never got about monorepos: Imagine you have an internal library and also two consumers of that library in the repo. But then you make breaking changes to the library but you only have time to update one of the consumers. Now how can the other consumer still use the old version of that library?

You don't make breaking changes. You provide the new API and the old API at the same time, and absorb the additional complexity as the library owner. Best case scenario everyone migrates to the new API and eventually remove the old one. This sounds onerous, but keep in mind at a certain scale there is no one commit in production at any given time. You could never roll out an atomic breaking change anyway, so going th…

Thank you for the response!

Genuine question: if you can't have one commit in production at any given time, what advantages for the monorepo remain?

Re: The Ingredients of a Productive Monorepo

#217

Post by matthew16550 »

przmk wrote: post #82
lxe wrote: post #8

So there are 2 kinds of big tech monorepos. One is the kind described in the article here: "THE" monorepo of the (mostly) entire codebase, requiring custom VCS, custom CI, and a team of 200 engineering supporting this whole thing. Uber and Meta and I guess Google do it this way now. It takes years of pain to reach to this point. It usually starts with the other kind of "monorepo": The other kind is the "multirepo mon…

At my current $dayjob, there is a backend that is split into ~11 git repos which results in a single feature being split among 4-5 merge requests and it's very annoying. We're about to begin evaluating monorepos to group them all (among other projects). What would the alternative to a monorepo be in this case, knowing that we can't bundle the repos together?

As an asside, I've found IntelliJ very helpful in this situation as it can load many repos into one project then doing commits / pushes / branches etc across various repos at the same time just seemed to work the way I wanted without much thinking about it.

Re: The Ingredients of a Productive Monorepo

#218

Post by wdb »

przmk wrote: post #82
lxe wrote: post #8

So there are 2 kinds of big tech monorepos. One is the kind described in the article here: "THE" monorepo of the (mostly) entire codebase, requiring custom VCS, custom CI, and a team of 200 engineering supporting this whole thing. Uber and Meta and I guess Google do it this way now. It takes years of pain to reach to this point. It usually starts with the other kind of "monorepo": The other kind is the "multirepo mon…

At my current $dayjob, there is a backend that is split into ~11 git repos which results in a single feature being split among 4-5 merge requests and it's very annoying. We're about to begin evaluating monorepos to group them all (among other projects). What would the alternative to a monorepo be in this case, knowing that we can't bundle the repos together?

Only 11 repos? I am at 76 repos for one backend. lol It's madness.

Re: The Ingredients of a Productive Monorepo

#219

Post by peterldowns »

mikn wrote: post #140
lbhdc wrote: post #124

Earlier quoted context omitted.

I am in a pretty similar situation as you, and have really been feeling the benefits of going all in on bazel. > We have made this tool layer which means if I run `go` or `kubectl` while in our repo, it's built and provided by Bazel itself. This means that all of us are always on the same version of tools, and we never have to maintain local installations. Currently I have to run `bazel run `. Your solution sounds wa…

Hi! Previously mentioned ex-Google SRE! There are a few layers to it - to make it work "ok" you need to first have a tool runner wrapper rule that does something similar to: ``` ctx.actions.write(output=""" tool_path=$(realpath {tool_short_path}) cd ${{BUILD_WORKING_DIRECTORY}} exec $tool_path """.format(tool_short_path=tool.short_path) ``` The purpose of this rule is to ensure that the tool's CWD is actually where y…

Any chance you'll be releasing your rules? I'd love to see how you do it.

Re: The Ingredients of a Productive Monorepo

#220

Post by bluGill »

cloogshicer wrote: post #210
bluGill wrote: post #101

Earlier quoted context omitted.

it is the only sane thing to do. Allowing everyone to use their own fork means when a major bug is found you have to fix thousands of forks. If the bug is a security zero day you don't have time.

Couldn't you just leave the other consumer at the old release (presumably well tested, stable)? I don't see how being forced to upgrade all consumers is a good thing.

Mapbe but each one now is another thing that you need to fix if a major issue is found. If it is only a few releases not a problem but it can get to hundreds and that becomes hard. Particularly if the fix can be cherry-picked cleanly to other branches.
Post reply on HN