Monorepos: Please don’t • Hacker News Board

Re: Monorepos: Please don’t

#331

Post by jogjayr » Thu, Jan 03, 2019, 5:29 PM UTC

marcinzm wrote: post #253

jogjayr wrote: post #102

Earlier quoted context omitted.

> You can always just put all your small repos into a big one. It's not quite as simple as that. You'll need to avoid rebuilding the entire repo for every change - using something like Bazel. This means your build tooling has to be replaced entirely, which is a non-trivial task and not something your devops/release engineering team will thank you for. For any 3rd party libraries used by your projects you need to eith…

>not something your devops/release engineering team will thank you for. It's their job. If they actively don't want to do work then you probably made a hiring mistake somewhere. By that logic what DevOps really wants is to the company to shut down since then they'd have none of that tedious work to do.

Release engineering is a conservative profession. They won't thank you for upsetting all their established processes, introducing new software and systems that have to be maintained, and kept running just because you don't like how your code is laid out.

It's harder to make a business value case for this type of change - there are only vaguely worded promises of "improved developer velocity". Contrast that with a change that automates or makes faster some aspect of building and releasing - a professional release engineering team would be all over it because they can demonstrate value in that work.

In any company beyond 50 people, there are multiple engineering managers, directors or the VP of engineering that will need to back this initiative to make the release engineering team do it. It's really not as simple as "dump all the code in one repo". I'm speaking from experience.

Re: Monorepos: Please don’t

#332

Post by bluGill » Thu, Jan 03, 2019, 5:30 PM UTC

marcinzm wrote: post #328

bluGill wrote: post #326

Earlier quoted context omitted.

They will have to work the entire Christmas break to pull off the switch. That will not make them happy. Rolling out massive changes like this is not easy, it needs to be planed in advance, tools built, dry-runs run, and then the final move. It also requirement management to not schedule any releases for several weeks before or after the change. As a member of a tool team I wouldn't think of doing this level of chang…

You're making a lot of assumptions about how such a move would be done which I don't feel are warranted. You're picking the hardest most painful option and then using it to claim the process is painful rather than that the option you chose is painful. If I was moving many small repos into a single mono repo then I'd do it one repo at a time. Presumably your small repos are independent entities so there's no reason to…

Now I lose lots of weekends making each separate move.

Re: Monorepos: Please don’t

#333

Post by joshuamorton » Thu, Jan 03, 2019, 5:31 PM UTC

thanatos_dem wrote: post #289

yowlingcat wrote: post #256

I think this article is complete horseshit. A monorepo will serve you 99% of the time until you hit a certain level of scale when you get to worry about whether a monorepo or a polyrepo is actually material. Most cases are never going to get there. Before that point, a polyrepo is purely a distraction and makes synchronous deployment really painful. We had to migrate a polyrepo to a monorepo and it was not fun becaus…

I work on CI/CD systems, and that’s one thing that definitely gets harder in a monorepo. So you made a commit. What artifacts change as a result? What do you need to rebuild, retest, and redeploy? It doesn’t take a large amount of scale to make rebuilding and retesting everything impossible. In a poly repo world, the repository is generally the unit of building and deployment. In monorepo it gets more messy. For inst…

bazel/buck/pants all solve this, but independently of that, they're probably the best build systems.

Re: Monorepos: Please don’t

#334

Post by erik_seaberg » Thu, Jan 03, 2019, 5:31 PM UTC

rossjudson wrote: post #163

erik_seaberg wrote: post #3

I wish this had touched on polyrepos' ability to pin known-good versions of dependencies; that tends to be the Achilles' heel of monorepos.

Hmm. Monorepos strongly favor "always-good". Core library breakage gets detected and rolled back really fast. It's been months since I've been affected by one. Known-good is in the eye of the beholder, and is just another dimension that generates breakage.

google3 breaking was a bimonthly event that left the entire team aground. Bad enough that it happened, but the worst part is that we couldn't do anything about it but wait.

Re: Monorepos: Please don’t

#335

Post by jogjayr » Thu, Jan 03, 2019, 5:32 PM UTC

adrianN wrote: post #118

jogjayr wrote: post #102

Earlier quoted context omitted.

> You can always just put all your small repos into a big one. It's not quite as simple as that. You'll need to avoid rebuilding the entire repo for every change - using something like Bazel. This means your build tooling has to be replaced entirely, which is a non-trivial task and not something your devops/release engineering team will thank you for. For any 3rd party libraries used by your projects you need to eith…

Eh, if you small repos build separately before you put them into one big repository, they'll build separately after, even if you just have a Makefile in each. If building your software depended on building its parts first, you already have the tooling to do so.

This really does not work for all languages. Speaking from personal experience, trying to monorepo multiple JS projects managed by NPM, it's more work than that.

Re: Monorepos: Please don’t

#336

Post by marcosdumay » Thu, Jan 03, 2019, 5:33 PM UTC

fnord123 wrote: post #244

EngineerBetter wrote: post #95

Visited a customer recently who had inherited a monorepo. All their CI and release problems traced back to it. At the risk of sounding like an old git, package coupling and package cohesion principles were defined for a reason. I do feel like a lot of patterns in contemporary development are kneejerk reactions to how last generation's programmers did things. Exceptions? Nah, multiple returns! Dependency management? W…

>Exceptions? Nah, multiple returns! Dependency management? Who needs it... Oh, wait. I thought consensus was that exceptions, like OOP, is an antipattern. I guess there's room for different opinions. :-/

There is a largely held opinion (that I share) that exceptions are a dated pattern that is better replaced by the more modern alternatives.

They are still much better than the older patterns they were created to replace, like multiple return.

Re: Monorepos: Please don’t

#337

Post by malkia » Thu, Jan 03, 2019, 5:34 PM UTC

malkia wrote: post #69

I do really like mono-repos, but google's other significant new project: fuchsia - is set-up as multi-git repo (and I believe chromium too, maybe android (haven't checked)). For fuchsia, they use a tool called "jiri"[1] to update the repos, previously (and maybe still in use) is the "gclient" sync tool [2] way from depot_tools[3] [1] - https://fuchsia.googlesource.com/jiri/ [2] - https://chromium.googlesource.com/chr…

"Bazel is getting there to support monorepos" -> support multiple repos (was tired when I wrote this I guess)...

Re: Monorepos: Please don’t

#338

Post by marcinzm » Thu, Jan 03, 2019, 5:34 PM UTC

bluGill wrote: post #332

marcinzm wrote: post #328

Earlier quoted context omitted.

You're making a lot of assumptions about how such a move would be done which I don't feel are warranted. You're picking the hardest most painful option and then using it to claim the process is painful rather than that the option you chose is painful. If I was moving many small repos into a single mono repo then I'd do it one repo at a time. Presumably your small repos are independent entities so there's no reason to…

Now I lose lots of weekends making each separate move.

Why weekends? Small moves means you do it during the week and during regular working hours. Done in branches with CI support so that it's pretty unlikely to break anything. If doing regular releases require you to be spending your weekends then you have bigger issues to fix first. Spread it over however long it takes. Why are you trying to make your life more difficult?

Re: Monorepos: Please don’t

#339

Post by softawre » Thu, Jan 03, 2019, 5:34 PM UTC

jskaggz wrote: post #320

yowlingcat wrote: post #256

I think this article is complete horseshit. A monorepo will serve you 99% of the time until you hit a certain level of scale when you get to worry about whether a monorepo or a polyrepo is actually material. Most cases are never going to get there. Before that point, a polyrepo is purely a distraction and makes synchronous deployment really painful. We had to migrate a polyrepo to a monorepo and it was not fun becaus…

Here here yowlingcat. Article is a way too prescriptive and agreed, borders on irresponsible. Monorepo vs polyrepo argument is way too broad a subject to create generalized stereotypes like this. These opinions sadly are taken as facts by impressionable managers, new developers, etc, and have cascading effects on the rest of us in the industry. Use what makes sense in the project environment and team, don't just thro…

It's interesting that you talk about "team's using monorepos". I think that's different than what the article is arguing against, which is an entire company (100+ devs) using a monorepo.

A team with 5 services and a web front-end in a single repo is doable with regular git. It's a different beast I think.

Re: Monorepos: Please don’t

#340

Post by jogjayr » Thu, Jan 03, 2019, 5:35 PM UTC

jhasse wrote: post #130

jogjayr wrote: post #107

Earlier quoted context omitted.

> Monorepos allow you to do very bad hacks (I need this other component over there; let me just put in a Symlink. Done.). Why would you put in a symlink? You could just provide a path to the actual component and import it into your project. > the worst thing you can get that you'll have to assemble multiple distinct, well-encapsulated (in terms of project structure) things into one When you have multiple repos, you a…

> Why would you put in a symlink? You could just provide a path to the actual component and import it into your project. Where do I need to put the path again? Ah what the heck, I'll just add a symlink inside a folder that's already somewhere in the build definitions.

Could a VCS simply blacklist symlink files? Plus if you have developers doing crap like this (and their colleagues letting it slide in code reviews) you have problems that can't be solved by monorepo vs polyrepo. You have an engineering culture problem.