My last 2 jobs have been working on developer productivity for 100+ developer organizations. One is a monorepo, one is not. Neither really seems to result in less work, or a better experience. But I've found that your choice just dictates what type of problems you have to solve. Monorepos are going to be mostly challenges around scaling the org in a single repo. Polyrepos are going to be mostly challenges with coordi…
Monorepos: Please don’t
251–260 of 402 posts
Re: Monorepos: Please don’t
#252Earlier quoted context omitted.
> Why would you put in a symlink? You could just provide a path to the actual component and import it into your project. Where do I need to put the path again? Ah what the heck, I'll just add a symlink inside a folder that's already somewhere in the build definitions.
What language and build tool is this that you're using? I don't know anyone who has abused Maven or Cargo or Go like this. And I don't imagine Visual Studio Solutions for C# are used like this. Is there an underlying disagreement based on JS/Ruby/Python scriptish coding (which creaks when a lot of developers work on it) vs C and C++ (which have astonishingly bad build system stories) vs big-iron languages that don't…
Re: Monorepos: Please don’t
#253Earlier quoted context omitted.
You can't split monorepos after the fact, at least not without immense costs. You can always just put all your small repos into a big one.
> You can always just put all your small repos into a big one. It's not quite as simple as that. You'll need to avoid rebuilding the entire repo for every change - using something like Bazel. This means your build tooling has to be replaced entirely, which is a non-trivial task and not something your devops/release engineering team will thank you for. For any 3rd party libraries used by your projects you need to eith…
It's their job. If they actively don't want to do work then you probably made a hiring mistake somewhere. By that logic what DevOps really wants is to the company to shut down since then they'd have none of that tedious work to do.
Re: Monorepos: Please don’t
#254Earlier quoted context omitted.
And no, you cannot split a monorepo into a polyrepo easily. Been there, done that. _The reason is that working in a monorepo versus multiple repos influences the architecture quite a lot and the monorepo leads to very unclear boundaries. I think you are conflating a monorepo (where boundaries can still be established, e.g. via a module isolation mechanism specific to the stack used) with a "monoproject"/"monomodule",…
If the projects within the monorepo are decoupled and have clear boundaries then why not have them in separate repositories?... In my opinion monorepos make refactoring dependant projects much easier. However it is much harder to establish and enforce clear boundaries...
So when someone only wants a submodule they can happily only clone that, but when someone wants all stuff (which is the default case), the can clone and install all at once.
Downside is that I have to commit twice
Re: Monorepos: Please don’t
#255Earlier quoted context omitted.
> Because obviously the changes that you make in the gui are completely isolated from the changes you make on the server. In my experience, that is almost never the case. Often, the frontend requires a new endpoint or a modification to an existing endpoint. If you don't coordinate this change, you end up with a non-functional PR that cannot even be tested. Same happens when the backend proposes an endpoint change tha…
You make the endpoint first, and test it without the UI. What challenges do you foresee here?
* Any non-backwards compatible change in the interface between the components. Yes this can be solved. But when working in a smaller team on proprietary software why use time solving a problem you don't need to solve?
(This is from experience.)
Re: Monorepos: Please don’t
#256Re: Monorepos: Please don’t
#257Yes, monorepos can be slow to browse through if the VCS isn’t configured to handle the size (sparse pulls aren’t the default with Git; that alone can make a massive difference when your repo is massive). Polyrepos can be just as slow? however; what’s worse is that there are more of them.
I remember working with a repo that was >20GB large, mostly from videos (we didn’t know that initially). Pulling that repo took _forever_. Nobody on that team cared because they almost never did a fresh pull and accounted the time it took for their CI/CD to do so in their reports. If it were a monorepo, MANY teams would’ve felt that pain more immediately.
Yes, monorepos require some tooling to prevent a gazillion artifacts from being deployed at once (and to specify what’s related to what if code lives across different folders). So do polyrepos! I’ve configured a few Jenkins jobs for my clients to dynamically pull different co-dependent Git repositories at build time. It’s a pain! Especially when multiple credentials are involved! Then there’s the whole “We have a gazillion repos and 20% of them are junk” problem, which requires automated reaping; also a more difficult problem than it seems.
Same with refactors. Refactors across polyrepos are just as much of a pain because you’re now subject to n build and review processes/pull requests, and seeing the entire diff is hard/impossible. This introduces mistakes. If anything, refactors in polyrepos are more of an event than they are for monorepos.
While monorepos have their problems, I will continue to advocate for them because the ability to see what’s going on in one place and for any developer to propose changes to any part of the code (theoretically) is massively beneficial, ESPECIALLY for complex business domains like healthcare or financial services. Plus, you will have a RelEng/BuildEng team when your codebase and engineering org gets large enough; why add more complexity by creating a gazillion repos that are possibly related to each other?
(The large engineering organization without a team focussed on tools and builds doesn’t exist. If it doesn’t, that means that some/many developers are spending way more time spinning their wheels on build systems than they should be.)
The real reason why monorepos don’t happen in the aforementioned domains is because there’s no easy way to allow them and pass regulatory audits.
Many regulating bodies require hard boundaries enforced by role-based access control, especially for code that deals with personally-identifiable information or code between two or more domains that have a Chinese Wall between them. “All of my developers can check out the entire codebase” is an easy way to get fined hard, and polyrepos are much easier to restrict access into than folders in a monorepo are (one advantage not mentioned in the article). While you _can_ restrict access into directories within a single repo, doing so is not straightforward, and most organizations would rather not waste the engineering effort.
I would like to think that Google and Facebook have gotten away with it because they implemented a monorepo from the very beginning and the engineering involved in splitting it up is much more involved than engineering around it.
That said, I continue to advocate for them because discoverability is good and it builds a better engineering culture in the end. I would rather hit those walls and make just-in-time exceptions for them than assume that the walls are there and create a worse development experience without exploring better alternatives.
Re: Monorepos: Please don’t
#258My advice is that if components need to release together, then they ought to be in the same repo. I'd probably go further and say that if you just think components might need to release together then they should go in the same repo, because you can in fact pretty easily manage projects with different release schedules from the same repo if you really need to. On the other hand if you've got a whole bunch of component…
What are you talking about! In my perfect micro services world I just have these enforced bounded contexts that are so perfectly designed they never need to change. Consequently all parts of the system are perfectly independent snowflakes that can be deployed without thinking about any other parts of the system. It’s beautiful really when you think about the mess that things were before we could do this!
Re: Monorepos: Please don’t
#259Visited a customer recently who had inherited a monorepo. All their CI and release problems traced back to it. At the risk of sounding like an old git, package coupling and package cohesion principles were defined for a reason. I do feel like a lot of patterns in contemporary development are kneejerk reactions to how last generation's programmers did things. Exceptions? Nah, multiple returns! Dependency management? W…
> Wang it all in one, and then invent your own tooling to cope with it! Why don’t you believe that additional tooling is required to manage software whose sources are composed from multiple repositories?
Re: Monorepos: Please don’t
#260Earlier quoted context omitted.
Any is more than 0 though. In my experience (probably shared by many devs), polyrepos don’t require a team, or even a single person, dedicated to version control. It’s a minor part of the software management (usually: “mind if I create a new repo for this?” “Yes/no”). It does affect dependency management but no more than any external dependency.
A polyrepo setup at Google's scale would pretty obviously require some dev work. For example, their CI/build story would be way more complex.