Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Ingredients of a Productive Monorepo

blog.swgillespie.me

Post reply on HN
Print view View original

261–268 of 268 posts

Re: The Ingredients of a Productive Monorepo

#261

Post by boxed »

wocram wrote: post #238
boxed wrote: post #79

The article links to a site with this definition: > A monorepo is a single repository containing multiple distinct projects, with well-defined relationships. It would be better if there were terms that delineated "one repo for the company" from "one repo per project" from "many repos for a single project".

I think most monorepo advocates are actually anti "one repo per project" at heart. That's the real anti-pattern imo.

I don't get it. That's the only pattern that makes any sense imo.

If you get problems because two or more projects needs to be updated simultaneously because of DRY or whatever, I would argue that you actually have ONE project and you're just fooling yourself.

Hmm.. actually, I guess that's what you're saying? When I said "project" above, I meant something like "product" or "system".

Re: The Ingredients of a Productive Monorepo

#262

Post by chrismatic »

oftenwrong wrote: post #253
chrismatic wrote: post #76

The point about trying to stick with a single language build tooling really cannot be stressed enough. It is what prompted me to write a simplified version of Bazel, a generic "target determinator" with caching capabilities if you will. I call it "Grog", the monorepo build tool for the grug-brained developer. https://grog.build/why-grog/

I am excited to learn of this project. I started working on something quite similar recently. It's a surprisingly unaddressed niche. One thing your tool appears to be missing (IMO) is execution sandboxing. This is useful, as you likely know, for avoiding undeclared dependencies and for avoiding dirty builds due to actions polluting the source directory, among other things. I was playing around with allowing configura…

I fully agree. I was also thinking about docker or symlinks, but it seemed hard to design the API without any actual user feedback. Pants environments [0] look interesting.

Very cool that you are also recognizing this issue and working on it. I sent you an email in case you want to exchange further.

[0] https://www.pantsbuild.org/stable/docs/introduction/welcome-...

Re: The Ingredients of a Productive Monorepo

#263

Post by vinnymac »

wocram wrote: post #239
vinnymac wrote: post #20

I established monorepos for the last two large projects I operated. I’ve never heard such nice compliments from contributors in my whole career. It seems not only can it be a productivity booster but people genuinely love when things are easy to grok and painless. Multiple large monorepos in an organization are highly valuable imo, and should become more of a thing over time.

Why multiple manyrepos over a single monorepo?

It would require a blog post for me to answer this in detail, but overall it’s due to the fact that monorepos trend toward specific stacks and runtimes. They aren’t often as flexible as they’d have you believe, so you might find a low level programmer isn’t able to operate as efficiently when operating in a primarily NodeJS-based Monorepo. Plus it makes it easier to separate parts of your business that are distinct, simplifying the complexity of security, maintenance, and compliance.

I don’t think one should have 100s of them, practically speaking less than 5 should be enough to model any business today. It can be freeing though, and allows developers to take advantage of the tools that best fit there needs so they can go back to getting shit done.

Re: The Ingredients of a Productive Monorepo

#264

Post by scrubs »

The OP had a point to make then made it. It's refreshing. And, moreover, I'm smarter for it. Well done. Thank you for posting it. As readers may see from my recent comments elsewhere, there's a ton of junk out there. But when the good stuff arrives, one likewise stops, and says so.

Re: The Ingredients of a Productive Monorepo

#265

Post by scrubs »

codethief wrote: post #222
gengstrand wrote: post #184

Earlier quoted context omitted.

It is true that there are significant benefits to monorepo but it comes at a cost. Managing a monorepo is more expensive than polyrepo. For the details behind that claim, check out https://www.exploravention.com/blogs/monorepo/ The question is this. Do the costs of monorepo justify the benefits for your situation? The answer is not always yes.

Note that the costs depend on the scale and size of the monorepo, and a polyrepo is not without costs either (which people often like to forget or ignore because they are less visible). I spent some years at small to mid-sized companies (~30-100 devs) that would have profited from a monorepo. However, people were in the habit of splitting repositories every other month. Sometimes a single team would own a given repos…

@codethief Let's stipulate to all of what you write ... now per OP (to take one example) you'll need to cross the rubicon and fork git, or write your own SCS ... you're not using off the shelf.

What do you do? How do you successfully argue for the funds and time to do it then move all the poly repos into it?

Because until that's happened the problem isn't engaged in any meaningful way.

Re: The Ingredients of a Productive Monorepo

#266

Post by cloogshicer »

codethief wrote: post #248
cloogshicer wrote: post #216

Earlier quoted context omitted.

Thank you for the response! Genuine question: if you can't have one commit in production at any given time, what advantages for the monorepo remain?

> if you can't have one commit in production at any given time That might be possible in a simple library + 1 consumer scenario if you follow the other commentors' recommendation to always update library + consumer at once. But in many cases you can't, anyway, because you're deploying several artifacts or services from your monorepo, not just one. So while "1 commit in production at any given time" is certainly neat,…

Thank you very much for the info!

Re: The Ingredients of a Productive Monorepo

#267

Post by bluGill »

Too wrote: post #246
bluGill wrote: post #125

Earlier quoted context omitted.

I keep meaning to write a blog post... The short answer, start with a package management system like conan or npm (we rolled our own - releasing 1.0 the same month I first heard of conan which was then around version 0.6 - don't follow our example). Then you just need processes to ensure that everyone constantly has the latest version of all the repos they depend on - which ends up being a full time job for someone t…

> Then you just need processes to ensure that everyone constantly has the latest version of all the repos they depend on - which ends up being a full time job for someone to manage. One full time job equivalent can buy a lot of tooling. Tooling that not only replaces this role but also shifts the feedback a lot closer to dev introducing the breaking change.

I realize this is more than a week ago and nobody will see it, but...

Every large project has this position. Smaller projects it isn't a full time job and so they distribute it amoung the team members and don't track the cost. Larger projects it is too large for a full time person and so they are forced to distribute the costs of this and just cannot track the costs. We happen to be in the sweet spot where a full time person can do the job and so we can track that cost. However make no mistake everything this person is doing is done on every project.

I agree with tooling being important. the person we have doing this job is a great engineer who automates everything he can, but there is still a lot of work that needs to be done. Some of it cannot be automated. (many of the problems are people problems)

Re: The Ingredients of a Productive Monorepo

#268

Post by codethief »

scrubs wrote: post #265
codethief wrote: post #222

Earlier quoted context omitted.

Note that the costs depend on the scale and size of the monorepo, and a polyrepo is not without costs either (which people often like to forget or ignore because they are less visible). I spent some years at small to mid-sized companies (~30-100 devs) that would have profited from a monorepo. However, people were in the habit of splitting repositories every other month. Sometimes a single team would own a given repos…

@codethief Let's stipulate to all of what you write ... now per OP (to take one example) you'll need to cross the rubicon and fork git, or write your own SCS ... you're not using off the shelf. What do you do? How do you successfully argue for the funds and time to do it then move all the poly repos into it? Because until that's happened the problem isn't engaged in any meaningful way.

Why the need to fork git? It works just fine for a middle-sized company with a couple hundred devs. Let's not pretend every company is Google- or Meta-sized. Personally, I think the sweet spot for a monorepo is not a conglomerate-wide monorepo, anyway, but one at a much smaller scale where you bundle all those projects together that depend on each other but not more. There is little harm in keeping things in separate repos when they have nothing to do with each other. Yes, you might have to copy the tooling but 1) the tooling is not going to be special, 2) if you have to do the copying once or twice, that's not a big deal.
Post reply on HN