The Ingredients of a Productive Monorepo
blog.swgillespie.me
The Ingredients of a Productive Monorepo
1–10 of 268 posts
Re: The Ingredients of a Productive Monorepo
#2Good practical article, thank you. I've added the link to my monorepo-vs-polyrepo guide here: https://github.com/joelparkerhenderson/monorepo-vs-polyrepo/
Re: The Ingredients of a Productive Monorepo
#3This definitely tracks with my experience in big tech - managing large scale build systems ends up taking a team that works on the build system itself. The underlying repo technology itself needs to work at scale & that was with a virtual file system that downloaded source files on demand when you needed to access them.
One thing that this article didn't mention is that most development was done either on your development server running a datacenter (think ~50-100 cores) - or on an "on demand" machine that was like a short lived container that generally stayed up to date with known good commits every few hours. IDE was integrated with devservers / machines & generally language servers, other services were prewarmed or automatically setup via chef/ansible, etc. Rarely would you want to run the larger monorepos on your laptop client (exception would generally be mobile apps, Mac OS apps, etc.).
Re: The Ingredients of a Productive Monorepo
#4Working with a well maintained mono repo is so nice, any other workflow just sucks to go back to. Working with a "lets do a monorepo" monorepo, where who ever set it up didn't understand the points in this article and more is a nightmare.
I think this is a business opportunity, if someone could sell the polished monorepo experience and tools to companies with engineering organizations but can't pull off a successful "we need to fork git" project to support their developers.
Re: The Ingredients of a Productive Monorepo
#5Good practical article, thank you. I've added the link to my monorepo-vs-polyrepo guide here: https://github.com/joelparkerhenderson/monorepo-vs-polyrepo/
thanks!
Re: The Ingredients of a Productive Monorepo
#6This definitely tracks with my experience in big tech - managing large scale build systems ends up taking a team that works on the build system itself. The underlying repo technology itself needs to work at scale & that was with a virtual file system that downloaded source files on demand when you needed to access them. One thing that this article didn't mention is that most development was done either on your develo…
Yeah - I worked on that build team probably at the same place you did!
I think for a lot of users it's more important that the monorepo devenv be reproducible than be specifically local or specifically remote. It's certainly easier to pull this off when it's a remote devserver that gets regularly imaged.
Re: The Ingredients of a Productive Monorepo
#7Working with a well maintained mono repo is so nice, any other workflow just sucks to go back to. Working with a "lets do a monorepo" monorepo, where who ever set it up didn't understand the points in this article and more is a nightmare. I think this is a business opportunity, if someone could sell the polished monorepo experience and tools to companies with engineering organizations but can't pull off a successful…
It is a business opportunity, NX is offering it.
In my previous startup, I started developing from the get go with NX, it became a huge velocity boost to our team. With 15 person RND we had standards that a 100 person RND didn’t accomplish.
In my new company (which has bought the startup), they tried the “let’s do a monorepo” approach. It is a catastrophe.
I am now in the process of migrating them to NX with great results.
Re: The Ingredients of a Productive Monorepo
#8So there are 2 kinds of big tech monorepos.
One is the kind described in the article here: "THE" monorepo of the (mostly) entire codebase, requiring custom VCS, custom CI, and a team of 200 engineering supporting this whole thing. Uber and Meta and I guess Google do it this way now. It takes years of pain to reach to this point. It usually starts with the other kind of "monorepo":
The other kind is the "multirepo monorepo" where individual teams decide to start clustering their projects in monorepos loosely organized around orgs. The frontend folks want to use Turborepo and they hate Bazel. The Java people want to use Bazel and don't know that anything else really exists. The Python people do whatever the python people do these days after giving up on Poetry, etc... Eventually these might coalesce into larger monorepos.
Either approach costs millions of dollars and millions of hours of developers' time and effort. The effort is largely defensible to the business leaders by skillful technology VPs, and the resulting state is mostly supported by the developers who chose to forget the horror that they had to endure to actually reach it.
Re: The Ingredients of a Productive Monorepo
#9Working with a well maintained mono repo is so nice, any other workflow just sucks to go back to. Working with a "lets do a monorepo" monorepo, where who ever set it up didn't understand the points in this article and more is a nightmare. I think this is a business opportunity, if someone could sell the polished monorepo experience and tools to companies with engineering organizations but can't pull off a successful…
It is a business opportunity, NX is offering it. In my previous startup, I started developing from the get go with NX, it became a huge velocity boost to our team. With 15 person RND we had standards that a 100 person RND didn’t accomplish. In my new company (which has bought the startup), they tried the “let’s do a monorepo” approach. It is a catastrophe. I am now in the process of migrating them to NX with great re…
Likewise, we’re using NX at my company and it has been a great experience.
Previous mono repo experiences were nothing short of a nightmare so it’s refreshing to see tooling come so far.
Re: The Ingredients of a Productive Monorepo
#10Working with a well maintained mono repo is so nice, any other workflow just sucks to go back to. Working with a "lets do a monorepo" monorepo, where who ever set it up didn't understand the points in this article and more is a nightmare. I think this is a business opportunity, if someone could sell the polished monorepo experience and tools to companies with engineering organizations but can't pull off a successful…
It is a business opportunity, NX is offering it. In my previous startup, I started developing from the get go with NX, it became a huge velocity boost to our team. With 15 person RND we had standards that a 100 person RND didn’t accomplish. In my new company (which has bought the startup), they tried the “let’s do a monorepo” approach. It is a catastrophe. I am now in the process of migrating them to NX with great re…
They’re on the right path but still have a lot to learn in the testing department.
Source: my job’s monorepo is running nx, but I’m not in developer productivity; used to work with a large codebase with an accompanying test suite of thousands of hours of a single box and it’s kinda like watching people rediscovering the roundness required to make the wheel.