Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Ingredients of a Productive Monorepo

blog.swgillespie.me

Post reply on HN
Print view View original

81–90 of 268 posts

Re: The Ingredients of a Productive Monorepo

#81

Post by 946789987649 »

gorgoiler wrote: post #65

An unspoken truth of a monorepo is that everyone is committed to developing on trunk, and trunk is never allowed to be broken. The consequence of this is that execution must be configurable at runtime: feature flags and configuration options with old and new code alongside each other. You can have a monorepo and still fail if every team works on their own branch and then attempts to integrate into trunk the week befo…

I didn't know places still had quarterly releases. That seems to like the one to resolve rather than a mono repo.

Re: The Ingredients of a Productive Monorepo

#82

Post by przmk »

lxe wrote: post #8

So there are 2 kinds of big tech monorepos. One is the kind described in the article here: "THE" monorepo of the (mostly) entire codebase, requiring custom VCS, custom CI, and a team of 200 engineering supporting this whole thing. Uber and Meta and I guess Google do it this way now. It takes years of pain to reach to this point. It usually starts with the other kind of "monorepo": The other kind is the "multirepo mon…

At my current $dayjob, there is a backend that is split into ~11 git repos which results in a single feature being split among 4-5 merge requests and it's very annoying. We're about to begin evaluating monorepos to group them all (among other projects). What would the alternative to a monorepo be in this case, knowing that we can't bundle the repos together?

Re: The Ingredients of a Productive Monorepo

#83

Post by Kinrany »

DanielHB wrote: post #80
lxe wrote: post #8

So there are 2 kinds of big tech monorepos. One is the kind described in the article here: "THE" monorepo of the (mostly) entire codebase, requiring custom VCS, custom CI, and a team of 200 engineering supporting this whole thing. Uber and Meta and I guess Google do it this way now. It takes years of pain to reach to this point. It usually starts with the other kind of "monorepo": The other kind is the "multirepo mon…

My company has been moving towards having monorepos per language stack. Decent compromise

That sounds worse than either option. At that point put it all in one repo with a directory for each language.

Re: The Ingredients of a Productive Monorepo

#84

Post by mattmanser »

bittermandel wrote: post #73

I firmly believe that us at Molnett(serverless cloud) going for a strict monorepo built with Bazel has been paramount to us being able to make the platform with a small team of ~1.5 full-time engineers. We can start the entire platform, Kubernetes operators and all, locally on our laptops using Tilt + Bazel + Kind. This works on both Mac and Linux. This means we can validate essentially all functionality, even our Bo…

The question here is why are you using micro service pattern and k8s with 2 Devs. That pattern is not designed for that small scale operation and adds tons of completely unnecessary complexity.

And does it really matter what you go with when you've got 1.5 engineers?

It's a non-problem at that scale as both engineers are intimately aware of how the entire build process works and can keep it in their head.

At that scale I've done no repo at all, repo stored on Dropbox, repo in VCS, SVN, whatever, and it all still worked fine.

It really hasn't added anything at all to your success.

BTW, it's still common for developers to start entire repos on their own laptops with zero hassles in tons of dev shops that haven't been silly and used k8s with 2 developers.

In fact at the start of my career I worked with 10 or so developers the shitty old MS one where you had to lock files so no-one else can use them. You'd checkout files to allow you to change them (very different to git checkout), otherwise they'd be ready only on your drive.

And the build was a massive VB script we had to run manually with params.

And it still worked.

We got some moaning when we moved to SVN too at how much better the old system was. Which was ridiculous as you used to have to run around and ask people to unlock key files to finish a ticket, which was made worse as we had developer consultants who'd be out of office for days on end.

So then you'd have to go hassle the greybeard who had admin rights to unlock the file for you (although he wasn't actually that old and didn't have a beard).

Re: The Ingredients of a Productive Monorepo

#85

Post by williamdclt »

oivey wrote: post #75
CamouflagedKiwi wrote: post #45

Earlier quoted context omitted.

That's an optimistic take on what happens in the polyrepo setup. A common alternative (I suspect by far the more common one) is that changes are made to the common area but not propagated to downstream repos, which all end up pinned to different versions of the common repo and struggle to update once they get ~years out of date.

Yeah. My experience is that the teams managing shared repos tend to shift responsibility for integrating their changes onto their users. They then also more often make breaking changes because they’re insulated from the costs of those changes.

the obvious result of that is: the changes are often not integrated for ages, if ever. Which means at some point it becomes a problem and the cost to do the integration has become much higher.

Re: The Ingredients of a Productive Monorepo

#86

Post by bee_rider »

mattmanser wrote: post #84
bittermandel wrote: post #73

I firmly believe that us at Molnett(serverless cloud) going for a strict monorepo built with Bazel has been paramount to us being able to make the platform with a small team of ~1.5 full-time engineers. We can start the entire platform, Kubernetes operators and all, locally on our laptops using Tilt + Bazel + Kind. This works on both Mac and Linux. This means we can validate essentially all functionality, even our Bo…

The question here is why are you using micro service pattern and k8s with 2 Devs. That pattern is not designed for that small scale operation and adds tons of completely unnecessary complexity. And does it really matter what you go with when you've got 1.5 engineers? It's a non-problem at that scale as both engineers are intimately aware of how the entire build process works and can keep it in their head. At that sca…

Keeping code in Dropbox kinda sucks even with 1 or .5 developers though. That said, yeah, a regular old git or (I assume, never used it) svn seems fine.

Re: The Ingredients of a Productive Monorepo

#87

Post by KaiserPro »

One of the things not covered here is how to deal with versioning.

By default a monorepo will give you $current and nothing else.

A monorepo is not a bad idea, but you should think about either preventing breaking changes in some dependency killing the build globally, or have some sort of artefact store that allows versioned libraries (both have problems, you'll need to work out which is better for you. )

Re: The Ingredients of a Productive Monorepo

#88

Post by amelius »

DanielHB wrote: post #80
lxe wrote: post #8

So there are 2 kinds of big tech monorepos. One is the kind described in the article here: "THE" monorepo of the (mostly) entire codebase, requiring custom VCS, custom CI, and a team of 200 engineering supporting this whole thing. Uber and Meta and I guess Google do it this way now. It takes years of pain to reach to this point. It usually starts with the other kind of "monorepo": The other kind is the "multirepo mon…

My company has been moving towards having monorepos per language stack. Decent compromise

This will start to become a problem if the stacks need to communicate with each other using versioned protocols.

Re: The Ingredients of a Productive Monorepo

#89

Post by eadmund »

bittermandel wrote: post #73

I firmly believe that us at Molnett(serverless cloud) going for a strict monorepo built with Bazel has been paramount to us being able to make the platform with a small team of ~1.5 full-time engineers. We can start the entire platform, Kubernetes operators and all, locally on our laptops using Tilt + Bazel + Kind. This works on both Mac and Linux. This means we can validate essentially all functionality, even our Bo…

> a small team of ~1.5 full-time engineers

Yes, with one and a half FTEs you should only have a single repo.

My experience with Bazel has been extremely bad, but I don’t think that it should necessarily be avoided completely. It may actually have some value on extremely large multi-team projects. But for less than two FTEs it seems like massive overkill.

I believe that you could do what you need with Kind (and maybe Tilt?), without Bazel.

> We have made this tool layer which means if I run `go` or `kubectl` while in our repo, it's built and provided by Bazel itself. This means that all of us are always on the same version of tools, and we never have to maintain local installations.

Go kind of does that for you already, with go.mod. Since kubectl is a Go program, you could achieve that goal the same way.

> it has been crucial to have an ex Google SRE on the team

I wonder how many additional team members y’all could afford in return for an ex-Googler’s salary expectations.

I sincerely hope that y’all find the maintenance expense of Bazel to be worth it going forward. Hopefully you will!

Re: The Ingredients of a Productive Monorepo

#90

Post by eadmund »

bee_rider wrote: post #86
mattmanser wrote: post #84

Earlier quoted context omitted.

The question here is why are you using micro service pattern and k8s with 2 Devs. That pattern is not designed for that small scale operation and adds tons of completely unnecessary complexity. And does it really matter what you go with when you've got 1.5 engineers? It's a non-problem at that scale as both engineers are intimately aware of how the entire build process works and can keep it in their head. At that sca…

Keeping code in Dropbox kinda sucks even with 1 or .5 developers though. That said, yeah, a regular old git or (I assume, never used it) svn seems fine.

> Keeping code in Dropbox kinda sucks even with 1 or .5 developers though. That said, yeah, a regular old git or (I assume, never used it) svn seems fine.

What you do is store the git repo in Dropbox, and developers just use it as a remote. With backups, this could actually go a reasonably long time, although I personally wouldn’t suggest it.

Post reply on HN