Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Ingredients of a Productive Monorepo

blog.swgillespie.me

Post reply on HN
Print view View original

191–200 of 268 posts

Re: The Ingredients of a Productive Monorepo

#191

Post by yencabulator »

bob1029 wrote: post #62

This thread is reminding me of a prior one about complexity merchants. I am seeing a lot of sentiment that there is somehow a technical sacrifice by moving to a monorepo. This is absolutely ludicrous unless you fail to grasp the power of a hierarchical file system. I don't see how a big mess like CI/CD is made easier by spreading it out to more points of configuration. To me the whole point of a monorepo is atomic co…

> To me the whole point of a monorepo is atomic commits for the whole org.

The belief that a monorepo makes a change somehow more atomic is one of the traps.

From the article:

> The greatest power and biggest lie of the monorepo is that it is possible to make atomic commits across your entire codebase. [...]

> Your monorepo now contains many different deployable artifacts that deploy at different times. It is also technically possible to make, for example, a breaking change to a service’s interface, a service’s implementation, and the service’s clients all in one PR. However, this PR will break when you deploy it because you do not deploy your service and all of its clients atomically. While this is also possible in a world with many repositories, the requirement to do this change in multiple pull requests is often enough to remind engineers that breaking changes to a service contract are not safe to make.

> Your users must understand that your deployment system operates asynchronously with respect to what happens in the monorepo. Its primary interaction with the monorepo is to go and pick up the “latest” build artifacts for a particular service; everything else happens on timetables that are potentially not under your control and can happen arbitrarily far in the future.

> A common CI job in a monorepo is to validate service contracts and make sure that they are not broken unless the author deliberately intended to do so, and they are required to provide a justification as to why such a change is OK.

Re: The Ingredients of a Productive Monorepo

#192

Post by someone654 »

l5870uoo9y wrote: post #190

Separating out the database layer in a monorepo package was the best architectural decision I made this year. Now it is my default because at some point you either want to rebuild the existing app entirely or separate out services such as public API access that all need access to the same database.

Can you elaborate on this? I’m facing a similar decision in my org and think sharing a common database store sounds smart. With rules of course, like clear ownership of data, only one writer, etc.

Re: The Ingredients of a Productive Monorepo

#193

Post by aldanor »

DanielHB wrote: post #80
lxe wrote: post #8

So there are 2 kinds of big tech monorepos. One is the kind described in the article here: "THE" monorepo of the (mostly) entire codebase, requiring custom VCS, custom CI, and a team of 200 engineering supporting this whole thing. Uber and Meta and I guess Google do it this way now. It takes years of pain to reach to this point. It usually starts with the other kind of "monorepo": The other kind is the "multirepo mon…

My company has been moving towards having monorepos per language stack. Decent compromise

And then at some point your Rust people write a Python module in Rust via pyo3, and it has to be integrated into Python build system and Python type checkers, but also needs local rust crates as build dependencies and local python packages as runtime dependencies... hm.

Re: The Ingredients of a Productive Monorepo

#194

Post by lolinder »

amelius wrote: post #187
lolinder wrote: post #110

Earlier quoted context omitted.

Why can't you just use versioning in your external-to-the-monorepo APIs and use HEAD within the monorepo? Nothing about combining some projects into a monorepo forces you into dropping everything else we know about software release cycles.

The point is that it is more work.

More work than what? More work than sharing HEAD in the monorepo, certainly. But it's definitely not more work than versioning across multiple repos because it's literally the same thing. When you're exporting code from a monorepo you follow all the same patterns you would from a small single library repo.

Re: The Ingredients of a Productive Monorepo

#196

Post by marcosdumay »

bob1029 wrote: post #62

This thread is reminding me of a prior one about complexity merchants. I am seeing a lot of sentiment that there is somehow a technical sacrifice by moving to a monorepo. This is absolutely ludicrous unless you fail to grasp the power of a hierarchical file system. I don't see how a big mess like CI/CD is made easier by spreading it out to more points of configuration. To me the whole point of a monorepo is atomic co…

> The power of this is really hard to overstate

A really useful heuristics when you are designing programming environments is: the more power you give to a team of developers, the more problems you will have.

Technically, atomic commits are not more power, they are less. But it does empower the team to work with bad interfaces. And that's a power that creates problems.

Re: The Ingredients of a Productive Monorepo

#197

Post by nc0 »

For the people interested in a good VCS system to achieve such monorepos, have a look at Ark [0]. It works really well for huge codebases, it is really fast, faster than Perforce Helix, it has an ethical and respectful pricing scheme, with a self-hosting mentality. Also it's indie, which is typically better than greedy corporate.

[0]: https://ark-vcs.com

Re: The Ingredients of a Productive Monorepo

#198

Post by maccard »

shawabawa3 wrote: post #19
echelon wrote: post #12

Earlier quoted context omitted.

As a former IC at a large monorepo company, I preferred monorepos over polyrepos. It was the "THE" monorepo, and it made understanding the company's service graph, call graph, ownership graph, etc etc. incredibly clear. Crystal clear. Vividly so. Polyrepos are tribal knowledge. You don't know where anything lives and you can't look or discover it. Every team does their own thing. Inheriting new code is a curse. Code…

Every time I've been monorepos compares with polyrepos it's always "monorepo plus millions of dollars of custom tool engineering" vs "stock polyrepo" Why can't we add millions of dollars of tool engineering on top of polyrepos to get some of the benefits of monorepos without a lot of the pain? E.g. it wouldn't be too hard to create "linked" PRs across repos for changes that span projects, with linked testing infrastr…

> Every time I've been monorepos compares with polyrepos it's always "monorepo plus millions of dollars of custom tool engineering" vs "stock polyrepo"

Not quite - it's "vs stock polyrepo with millions of dollars of engineering effort in manually doing what the monorepo tooling does".

Re: The Ingredients of a Productive Monorepo

#199

Post by lenkite »

eddd-ddde wrote: post #183
lenkite wrote: post #37

Earlier quoted context omitted.

Not sure if I get it. If you are using a product like Github Enterprise, you are already quite aware of what other people are doing. You have a lot of visibility, source-code search, etc. If you have a CICD that auto-creates issues you already can detect breakages, incompatibilities, etc. State of the "code universe" being atomic seems like a single point of failure.

Imagine team A vendors into their repo team B's code and starts adding their own little patches. Team B has no idea this is happening, as they only review code in repo B. Soon enough team A stops updating their dependency, and now you have two completely different libraries doing the "same" thing. Alternatively, team A simple pins their dependency to team B's repo at hash 12345, then just, never updates... How is tea…

This is already caught by multi-repo tooling like Github today. If you vendor in an outdated version with security vulnerabilities, issues are automatically raised on your repo. Team B doesn't need to do anything. It is Team-A's responsibility to adopt to latest changes.

Re: The Ingredients of a Productive Monorepo

#200

Post by marcosdumay »

l5870uoo9y wrote: post #190

Separating out the database layer in a monorepo package was the best architectural decision I made this year. Now it is my default because at some point you either want to rebuild the existing app entirely or separate out services such as public API access that all need access to the same database.

> in a monorepo package

Hum... Does that phrase mean you don't use anything remotely similar to a monorepo?

Post reply on HN