Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Monorepo is great if you're really good

yosefk.com

Post reply on HN
Print view View original

101–110 of 159 posts

Re: Monorepo is great if you're really good

#101

Post by beirut_bootleg »

mr_tristan wrote: post #40
qznc wrote: post #37

Does anybody know some monorepo horror stories? I have heard plenty of people complain about their many-repo structure and wishing for a monorepo. I would like to hear some concrete story where a monorepo went wrong. This article is just abstract opinion.

Oh yes. #1 problem I've experienced at multiple places: delaying integration with tons of branching. Usually the PM or PO force everyones to using some vague "product version" they track for public releases. Places that use monorepos well tend to have very few branches (like, maybe 2 or 3), which have nothing to do with your public product versioning scheme, and instead use "branch by abstraction" to stay integrated.…

My experience is similar. The newly promoted CTO used a botched merge and delayed release to justify the move to a complex but half-assed monorepo branching strategy with minimal tooling and documentation, trashing any systems teams had in place. We've started botching releases, losing prospective clients, and now are in release limbo.

Re: Monorepo is great if you're really good

#102

Post by jupp0r »

There is also a way to simultaneously reap the drawbacks of monorepos and multirepos: consolidate numerous small repos into somewhat big monorepos and then don't merge everything into one big repo.

This way, you'll have to deal with

* difficult tooling

* dependency hell between the mono repos (which are now way more tightly coupled due to the dependency graph between them being denser)

* long living branches causing way more collateral damage as described in the original article

* cross-repo changes have become even harder for all the reasons above

You get all the bad things and avoid those advantages! Welcome to my world :)

Re: Monorepo is great if you're really good

#103

Post by shados »

humanrebar wrote: post #75
DanFeldman wrote: post #28

My team just switched to a monorepo. It's been only a few weeks, so I can't claim any results yet, but we've lived w/ the pain of poly-repo for long enough that we were ready to invest in a single repo. We've spent a lot of time building and iterating a unified ci/cd environment to support the new repo. Previously each project had it's own test/deploy/build/publish story and usually it's own jenkins project. Now, eac…

I suspect for that number of projects monorepos make a lot of sense. The major technology organizations we hear about usually have at least several monorepos, due to the legacies of acquisitions and mergers if nothing else. At the scale of thousands of subprojects, I am not entirely sure the benefits are as advertised. There will be support of subprojects forked to public github.com or gitlab.com if nothing else. And…

Going from 15 to 3 is definitely a different discussion from going from 7000 to At 15, it feels like it's kind of just a toss up. We have several thousand repos, and sometimes we see 5-10 of them that really should be grouped, and we do so. Sometimes we see 1 repo that has 5-10 projects in it, and we break them down. Whatever works.

But when the entire org is on a dozen project you're potentially in the worse of both worlds. Your repos aren't small enough or aligned with team ownership enough to really benefit from it. So its straight overheard.

Re: Monorepo is great if you're really good

#104

Post by galkk »

I've experienced 3 large codebases with different approaches:

1. Monorepo: Google3

2. Non monorepo, large pile of #@$: Microsoft Exchange

3. Non monorepo, Amazon

After working with them, from my personal experience, monorepo was the best. Yes, Google has ton of internal stuff and they could go away with not using much of external dependencies, but when everything works, it works like a charm. Convenience of defining protobufs/contracts, ease of reference them and ton of the things are given to you when you're in the system.

At Google I never felt that the system is hostile to you. It was extremely easy to start hacking something if you'd like to. Yes, it's not only monorepo, but the overall quality of the tools available, but monorepo is also quite a significant part of it.

Re: Monorepo is great if you're really good

#105

Post by mr_tristan »

the_gipsy wrote: post #96
mr_tristan wrote: post #40

Earlier quoted context omitted.

Oh yes. #1 problem I've experienced at multiple places: delaying integration with tons of branching. Usually the PM or PO force everyones to using some vague "product version" they track for public releases. Places that use monorepos well tend to have very few branches (like, maybe 2 or 3), which have nothing to do with your public product versioning scheme, and instead use "branch by abstraction" to stay integrated.…

Or you could just tell them to stay off versioning/release/branching discussions, even in a monorepo. All they need to know is what release/deploy has what features, and when the release happens.

Easy to say when you're starting from scratch, but in all of my cases, these were "pre-existing conditions", i.e., was in place before I joined, and it took a long time to get people to see the light. It gets real tricky when you're _not_ a manager, and you're basically asking other managers to stop having such a loud voice in things.

Just another case of "culture eats strategy for breakfast". Once people start using any particular versioning strategy, it becomes canon, even if it's a terrible way to organize.

Re: Monorepo is great if you're really good

#106

Post by [deleted] »

jayd16 wrote: post #71
bartread wrote: post #30

Earlier quoted context omitted.

> "...The simplicity you try to reach is a false simplicity." Also applies to some microservice architectures I've seen. People completely disregard the complexity (and overhead!) of the interactions between microservices.

This is a poor take away. Even in a monolith, you still want to separate your concerns, yes? Your code should be almost as abstracted in a monolith as they are in micro services. In microservices the code is just deployed across multiple instances. RPCs and local method calls both need to be fault tolerant and race condition free. As you break up datastores, transactions become more complex but certainly you had a sp…

[deleted]

Re: Monorepo is great if you're really good

#107

Post by the_gipsy »

mr_tristan wrote: post #105
the_gipsy wrote: post #96

Earlier quoted context omitted.

Or you could just tell them to stay off versioning/release/branching discussions, even in a monorepo. All they need to know is what release/deploy has what features, and when the release happens.

Easy to say when you're starting from scratch, but in all of my cases, these were "pre-existing conditions", i.e., was in place before I joined, and it took a long time to get people to see the light. It gets real tricky when you're _not_ a manager, and you're basically asking other managers to stop having such a loud voice in things. Just another case of "culture eats strategy for breakfast". Once people start using…

Yeah sorry if I came off as dismissive. It's usually a long process, if that's how things run.

Re: Monorepo is great if you're really good

#108

Post by joshschreuder »

DanFeldman wrote: post #28

My team just switched to a monorepo. It's been only a few weeks, so I can't claim any results yet, but we've lived w/ the pain of poly-repo for long enough that we were ready to invest in a single repo. We've spent a lot of time building and iterating a unified ci/cd environment to support the new repo. Previously each project had it's own test/deploy/build/publish story and usually it's own jenkins project. Now, eac…

How do you handle building changes to just one of those projects? Can Jenkins do that (easily)?

I think that's the big thing that always puts me off monorepo... We'd basically be going from ten 5 minute builds to one 50 minute build if it wasn't possible to do incremental builds. IIRC Google and MS have purpose built tools that do impact detection to work out what to build for their monorepos to keep build times down.

Re: Monorepo is great if you're really good

#109

Post by gotofritz »

bartread wrote: post #30
jasim wrote: post #29

Linus Torvalds said something about this, in relation to microkernels. But the gist is in how interactions between many pieces makes the whole thing complex. Here's the quote from his book "Just for Fun". "The theory behind the microkernel is that operating systems are complicated. So you try to get some of the complexity out by modularizing it a lot. The tenet of the microkernel approach is that the kernel, which is…

> "...The simplicity you try to reach is a false simplicity." Also applies to some microservice architectures I've seen. People completely disregard the complexity (and overhead!) of the interactions between microservices.

Micro services are a different kettle of fish though

Firstly, the benefits they offer have often little to do with the architecture itself, but with the bigger picture (separating teams, CIs, allowing different stacks, managing costs, scaling, etc).

Secondly, unlike microkernels, not all microservices have to talk to every single other microservice. If you have a service to send emails, say, there'll be a few services that interact with it, but the majority won't. The same for an image resizing service.

So what you say doesn't necessarily hold

Re: Monorepo is great if you're really good

#110

Post by gotofritz »

gen220 wrote: post #53
vinay_ys wrote: post #52

Earlier quoted context omitted.

Does anyone here do Microservices well? And keep them in a monorepo?

We (~150 eng) build microservices in a polyglot environment (mostly Python, JS, and Go), all in a monorepo! We also build + deploy in containers with Jenkins, etc. The structure looks something like this: |-- third_party (third party libs) `-- python |-- libs (internally-written, common dependencies) `-- application_name |-- client (react app, connects to flask) |-- server (flask app, connects to services) `-- servic…

Organising apps by language seems weird...
Post reply on HN