Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Advantages of monolithic version control

danluu.com

Post reply on HN
Print view View original

81–90 of 144 posts

Re: Advantages of monolithic version control

#81

Post by jschrf »

I think that the question of whether or not to use a monorepo is not a function of "how to organize code for efficient coding". It's more of an actual organizational question; how and how often do you deploy, what are the actors and their needs, and what's the domain? Are you producing statically linked C libs? interrelated NPM packages? SaaS?

If you look at monorepos from a package management standpoint, they are usually just high-level graphs of dependencies. Or, more accurately, sources of dependencies. How these are rolled up, shipped, and ultimately deployed is a function of the operational culture and business needs more than it is source control or even language choices, in my opinion. Business needs impact source control in any sufficiently complex, source-controlling org.

That's not to say that, for example, small companies benefit from monorepos, while large ones benefit from small repos and packages (or the opposite). I think the pros and cons are entirely decoupled from codebase size and complexity. In order to do one or the other well, you need the right business needs, operational parameters, engineering culture, and resources. So, I always find it interesting to read about Google or Microsoft leveraging one, the other, or both approaches with their own codebases.

I liken the monorepo vs small packages approach to be a little bit like rendering a frame on a CPU vs a GPU. Do you build/test/deploy each "frame" (iteration) as a top-down, more-or-less-discrete block of work, or can you parallelize it and "ship" multiple compatible streams at once?

I suggest it depends almost entirely on the problem space and the "hardware" (business needs), far more than it does the actual code or volume of code.

Perhaps Conway's law here applies here in a sense, i.e. any organization that manages source code will produce a source control management scheme that is representative of how they deploy to downstream consumers.

Re: Advantages of monolithic version control

#82

Post by sa46 »

fishywang wrote: post #65
humanrebar wrote: post #7

Earlier quoted context omitted.

> Namely, it's very easy to depend on other code that the organization has created. On the other hand, I've seen the other sides of this in monorepos: 1. It's too easy to depend on code, so there is dependency bloat when something simpler would work just as well. 2. It's relatively hard to depend on things not in the repo, reinforcing not-invented-here culture.

2 is simple. Import everything you need to depend on into the repo. Google has 3rdparty directory in its mono repo to put them.

A large chunk of third party usage is open source Google libraries like Guava. Anecdotally, I don't end up using that many libraries from third party.

As for JavaScript, the hassle of importing the entire transitive closure of an NPM library you want into third party means it's much more attractive to go with NIH syndrome. I looked at importing ESLint, but it has something like 110 dependencies.

Re: Advantages of monolithic version control

#83

Post by remus »

jayd16 wrote: post #78

I personally find multi-repo thinking leads to better architecture. Cross project change history seems nice but if the projects are that coupled in the first place, why are they difference solutions to begin with? If you're building decoupled code you shouldn't need cross project changes. That said, I understand the worth of getting things done at the expense of rigor so I chalk this topic of discussion up to persona…

> If you're building decoupled code you shouldn't need cross project changes.

That's the theory, but in practice designing robust, future proof APIs has proven to be really hard in a lot of cases. You're then left with the option of supporting old APIs forever or migrating dependent code to new APIs, both of which are difficult in their own ways.

Re: Advantages of monolithic version control

#84

Post by jayd16 »

remus wrote: post #83
jayd16 wrote: post #78

I personally find multi-repo thinking leads to better architecture. Cross project change history seems nice but if the projects are that coupled in the first place, why are they difference solutions to begin with? If you're building decoupled code you shouldn't need cross project changes. That said, I understand the worth of getting things done at the expense of rigor so I chalk this topic of discussion up to persona…

> If you're building decoupled code you shouldn't need cross project changes. That's the theory, but in practice designing robust, future proof APIs has proven to be really hard in a lot of cases. You're then left with the option of supporting old APIs forever or migrating dependent code to new APIs, both of which are difficult in their own ways.

That has nothing to do with requiring an atomic cross project change. If your depended library needs to be updated, update as needed.

Upgrading other projects to support the new dependency version can come in a different commit. You only run into trouble when you've set up your projects to always use the latest version of their dependencies. That's a recipe for disaster.

Re: Advantages of monolithic version control

#86

Post by kingosticks »

How do people handle the case where in a large repository, with people committing almost constantly, pushing your changes to a git server on the other side of the world becomes quite tricky. By the time my push gets to the server it seems someone has gotten in before me and my repo is out of date. I have to pull the new changes and try again. During busy parts of the day (my afternoon, the U.S morning) you might have to loop this process a few times. If I was to check the code still builds after each of those merges I'd be there all day, so there is some risk there. When we had cvs this wasn't an issue since most changes are to completely different parts of the code base so you don't need their changes.

Re: Advantages of monolithic version control

#87

Post by scrollaway »

markild wrote: post #75
lloeki wrote: post #73

Earlier quoted context omitted.

For a close enough real life example, and very often just one file (PKGBUILD), ArchLinux uses one (orphaned) branch per package : https://git.archlinux.org/svntogit/packages.git/refs https://git.archlinux.org/svntogit/community.git/refs Orphaned branches allows you to have multiple independent trees in the same git repo, si it's basically a way to stuff many "repos" (as in history) in a single one (as in object stora…

That seems very weird. Do you happen to know the reasoning behind it?

Arch maintainer here. Arch Linux doesn't use git, it uses svn still. Each arch package-repository (core, extra, community...) is a single SVN repository. Packages are split in subdirectories in that repository and checked out individually. Every maintainer deals with a few dozen packages individually, not the whole. There are seldom any commits that span multiple repositories.

What you were linked is the "svn to git" mirror which has to somehow translate that aspect of the setup to git.

Incidentally, the way Arch svn is currently set up works really well for arch devs and it is very hard to find a good replacement to it using git. Having a monorepo split up in lots of microrepos is straight up not possible in git.

Re: Advantages of monolithic version control

#88

Post by ambivalence »

kingosticks wrote: post #86

How do people handle the case where in a large repository, with people committing almost constantly, pushing your changes to a git server on the other side of the world becomes quite tricky. By the time my push gets to the server it seems someone has gotten in before me and my repo is out of date. I have to pull the new changes and try again. During busy parts of the day (my afternoon, the U.S morning) you might have…

You don't ever push straight to the repo. You enqueue your changeset to be pushed by a central system. At Facebook we call this "asynchronous landing" and you're right, before this became a thing about five years back, at certain times of day it was very tricky to push your changes out.

Once you switch to this model, you can do convenient things like landing straight from the review system (by a "Ship it" button), landing after some checks were successfully performed, and so on.

Incidentally, this is similar to how merging pull requests by rebasing works on GitHub.

Re: Advantages of monolithic version control

#90

Post by MaxBarraclough »

kingosticks wrote: post #86

How do people handle the case where in a large repository, with people committing almost constantly, pushing your changes to a git server on the other side of the world becomes quite tricky. By the time my push gets to the server it seems someone has gotten in before me and my repo is out of date. I have to pull the new changes and try again. During busy parts of the day (my afternoon, the U.S morning) you might have…

One solution is to coordinate who 'owns' master, mutex style, using something like an intranet wiki.

When you 'hold the lock', you get to rebase and push, and 'release the lock', and if you've any sense the system will automatically ensure you haven't broken the build.

Of course, it's important not to waste time, as this serializes the commit process, as it were.

Post reply on HN