Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Ingredients of a Productive Monorepo

blog.swgillespie.me

Post reply on HN
Print view View original

131–140 of 268 posts

Re: The Ingredients of a Productive Monorepo

#131

Post by eddd-ddde »

lbhdc wrote: post #124
bittermandel wrote: post #73

I firmly believe that us at Molnett(serverless cloud) going for a strict monorepo built with Bazel has been paramount to us being able to make the platform with a small team of ~1.5 full-time engineers. We can start the entire platform, Kubernetes operators and all, locally on our laptops using Tilt + Bazel + Kind. This works on both Mac and Linux. This means we can validate essentially all functionality, even our Bo…

I am in a pretty similar situation as you, and have really been feeling the benefits of going all in on bazel. > We have made this tool layer which means if I run `go` or `kubectl` while in our repo, it's built and provided by Bazel itself. This means that all of us are always on the same version of tools, and we never have to maintain local installations. Currently I have to run `bazel run `. Your solution sounds wa…

Probably something similar to a py venv that you run to activate some aliases in your environment?

Re: The Ingredients of a Productive Monorepo

#132

Post by bluGill »

eddd-ddde wrote: post #127
cousin_it wrote: post #104

I've worked for a company with a large monorepo. At first I was a fan, but now I'm not so sure. The web of dependencies was too much. Now I think teams should be allowed to reuse other teams' code only as libraries or APIs with actual release cycles. There shouldn't be any "oh let's depend on the HEAD of this random build target somewhere else in the monorepo". There should be only "let's depend on a released version…

This just creates tons of fragmentation. The second you have multiple teams depending on multiple versions you are doomed. You are stuck maintaining multiple versions, with their own quirks and bugs. I think the one version rule is the most important part for a healthy monorepo.

I think one version is important for a healty polyrepo as well. You have to set lines where you say no new features unless you are all up to date. You can allow bug fix only releases to stay behind, but if you write a new feture it must be against the current latest of everything.

Otherwise you are doomed because there are so many different versions of everything in use. Some day a zero-day issue will hit all your projects as the same time and you will need months to get each in use version fixed.

Re: The Ingredients of a Productive Monorepo

#133

Post by [deleted] »

countWSS wrote: post #94

From viewpoint of security and separation of concerns giving unlimited access to everything by virtue of "everything" being stored in one giant repo sounds exceptionally short-sighted. A single rogue actor would be able to insert code to any component of choice instead of working on isolated repo with people who specifically know it and approve the code: the monorepo is a "big ball of mud" with vague shared responsib…

[deleted]

Re: The Ingredients of a Productive Monorepo

#134

Post by eddd-ddde »

bluGill wrote: post #100
spankalee wrote: post #54

Earlier quoted context omitted.

The whole point of a monorepo is to force you to update all of the consumers, and to realize that breaking changes are expensive. The two monorepo ways to do this: 1. Use automated refactoring tools that now work because it's one repo 2. Add the new behavior, migrate incrementally, then remove the old behavior

Both of those work in polyrepo. You need a tools team to make it happen though, just like monorepo needs a tool team. The tools needed are different but you still need them.

In a polyrepo it is more common that the update simply happens, now repo A depends on v1 and repo B depends on v2, then a year has passed and repo A doesn't even remember they still depend on an old insecure library.

Re: The Ingredients of a Productive Monorepo

#135

Post by jeffbee »

lenkite wrote: post #37
AlotOfReading wrote: post #18

Earlier quoted context omitted.

One of the big advantages is visibility. You can be aware of what other people are doing because you can see it. They'll naturally come talk to you (or vice versa) if they discover issues or want to use it. It also makes it much easier to detect breakages/incompatibilities between changes, since the state of the "code universe" is effectively atomic.

Not sure if I get it. If you are using a product like Github Enterprise, you are already quite aware of what other people are doing. You have a lot of visibility, source-code search, etc. If you have a CICD that auto-creates issues you already can detect breakages, incompatibilities, etc. State of the "code universe" being atomic seems like a single point of failure.

GitHub search is insanely bad and it cannot do things like navigating to definitions between repos in an org.

Re: The Ingredients of a Productive Monorepo

#136

Post by peterldowns »

lbhdc wrote: post #124
bittermandel wrote: post #73

I firmly believe that us at Molnett(serverless cloud) going for a strict monorepo built with Bazel has been paramount to us being able to make the platform with a small team of ~1.5 full-time engineers. We can start the entire platform, Kubernetes operators and all, locally on our laptops using Tilt + Bazel + Kind. This works on both Mac and Linux. This means we can validate essentially all functionality, even our Bo…

I am in a pretty similar situation as you, and have really been feeling the benefits of going all in on bazel. > We have made this tool layer which means if I run `go` or `kubectl` while in our repo, it's built and provided by Bazel itself. This means that all of us are always on the same version of tools, and we never have to maintain local installations. Currently I have to run `bazel run `. Your solution sounds wa…

Not the OP but you can use tools like direnv + mise/asdf/nix so that every time a developer cd's into the monorepo, their shell environment loads a pinned, declaratively-configured set of dependencies and tools whose definitions are part of the monorepo.

The way I'd naively set up something like OP described would be to have direnv + nix flake deliver you a copy of bazelisk, and then have some custom shell scripts added to $PATH that alias `go = bazel run go`, `kubectl = bazel run kubectl` or whatever custom wrappers you want.

(Handwaving and I know the above isn't quite correct)

Re: The Ingredients of a Productive Monorepo

#137

Post by gorgoiler »

946789987649 wrote: post #81
gorgoiler wrote: post #65

An unspoken truth of a monorepo is that everyone is committed to developing on trunk, and trunk is never allowed to be broken. The consequence of this is that execution must be configurable at runtime: feature flags and configuration options with old and new code alongside each other. You can have a monorepo and still fail if every team works on their own branch and then attempts to integrate into trunk the week befo…

I didn't know places still had quarterly releases. That seems to like the one to resolve rather than a mono repo.

It’s more common than you think if you expand your view of release a bit. On the one hand you very much still have shrink-wrap software (for example, all firmware) that ships on a very slow cadence.

On the other hand even the big tech companies will only expose code paths very slowly and very conservatively. Meta’s Threads.app for example combined both a constant churn of innovation on master with a very measured gating of new features shipping to the public.

The best teams do indeed, as you say, ship and test finished builds on a weekly or daily basis even if the stuff that gets under the customers’ / users’ / clients’ noses appears on a far less regular basis. After all, any kind of severe bug could necessitate a release at any moment.

Re: The Ingredients of a Productive Monorepo

#138

Post by bluGill »

anon7000 wrote: post #69
zvr wrote: post #58

Genuine question, because I've never worked somewhere with a monorepo infrastructure: is it really "one repo for all code in the organization" or "one repo for everything related"? In my organization we have around 70k internal git repos (and an order of magnitude fewer public ones), but of course not everything is related to everything else; we produce many distinct software products. I can understand "collect every…

When you have N repos, you also have N ways of managing dependencies, N ways of doing local bin scripts and dev environment setups, N projects with various out of date & deprecated setups, N places to look when you need to upgrade a vulnerable dependency, N services which may or may not configure telemetry in a consistent way, N different CI & deployment workflows… It just gets very difficult to manage, especially if…

You CAN have n different whatever in a polyrepo - that doesn't mean you must. You can settled on a company wide package manager, CI system, build system and whatever. That is what my company has, while each repo has their own setup scripts, there are maybe 100 lines in each repo (and almost all of those lines are redundant and could be combined if I spent some time).

The above breaks down when we have third party code - since they don't follow our common patterns for building so they have to do something different. Bringin that into a monorepo would be just as different from everything else.

Re: The Ingredients of a Productive Monorepo

#139

Post by bluGill »

chrismatic wrote: post #76

The point about trying to stick with a single language build tooling really cannot be stressed enough. It is what prompted me to write a simplified version of Bazel, a generic "target determinator" with caching capabilities if you will. I call it "Grog", the monorepo build tool for the grug-brained developer. https://grog.build/why-grog/

If a single language is an option you are a small project that is not facing the problems people on large projects are facing. A monorepo will be easy for you without read the article and the lessons learned.

Come back when you have millions of lines of code, written over decades by hundreds (or thousands) of full time developers.

Re: The Ingredients of a Productive Monorepo

#140

Post by mikn »

lbhdc wrote: post #124
bittermandel wrote: post #73

I firmly believe that us at Molnett(serverless cloud) going for a strict monorepo built with Bazel has been paramount to us being able to make the platform with a small team of ~1.5 full-time engineers. We can start the entire platform, Kubernetes operators and all, locally on our laptops using Tilt + Bazel + Kind. This works on both Mac and Linux. This means we can validate essentially all functionality, even our Bo…

I am in a pretty similar situation as you, and have really been feeling the benefits of going all in on bazel. > We have made this tool layer which means if I run `go` or `kubectl` while in our repo, it's built and provided by Bazel itself. This means that all of us are always on the same version of tools, and we never have to maintain local installations. Currently I have to run `bazel run `. Your solution sounds wa…

Hi! Previously mentioned ex-Google SRE! There are a few layers to it - to make it work "ok" you need to first have a tool runner wrapper rule that does something similar to:

```

ctx.actions.write(output="""

tool_path=$(realpath {tool_short_path})

cd ${{BUILD_WORKING_DIRECTORY}}

exec $tool_path

""".format(tool_short_path=tool.short_path)

```

The purpose of this rule is to ensure that the tool's CWD is actually where you are inside the repository and not within the runfiles folder that Bazel prepared for you.

The second step is to set up a symlink target, similar to this:

```

#! /usr/bin/env bash

tool_name=$(basename $0)

exec -a "$tool_name" bazel run --ui_event_filters=-info,-stdout,-stderr --noshow_progress //tools/bin:$tool_name -- "$@"

```

We need to filter out all UI events since for some tools we intercept (such as jq) it expects the stdout to be clean from other output when used programmatically.

We then create a symlink for each tool name (say kubectl) to this script from another folder, and then we use `direnv` to inject the folder of symlinks into the user's paths with an `.envrc` file in the repository root like this:

```

PATH=$PWD/tools/path:$PATH

```

We have had this in place for quite a while now - it does seem like this pattern has caught some more wind and buildbuddy.io has released a ruleset: https://github.com/buildbuddy-io/bazel_env.bzl paired with https://github.com/theoremlp/rules_multitool achieves the same thing that we have built internally, the main difference being that with the bazel run wrapper we have made, you always run the latest version, whereas with the bazel_env pattern you need to manually rerun their target to get the latest binaries. :)

Post reply on HN