Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Advantages of monolithic version control

danluu.com

Post reply on HN
Print view View original

21–30 of 144 posts

Re: Advantages of monolithic version control

#21

Post by StillBored »

chimeracoder wrote: post #11
amag wrote: post #6

On the other end of the spectrum, a colleague of mine recently told me about when his previous company (big, 100k+ employees) were starting to adopt git, some people seriously considered having a separate repository per file ! "Because then you can just set up your project as using version X of file A and version Y of file B" It's good we now have (at least) Google, Facebook and Microsoft as examples of companies usi…

> On the other end of the spectrum, a colleague of mine recently told me about when his previous company (big, 100k+ employees) were starting to adopt git, some people seriously considered having a separate repository per file! "Because then you can just set up your project as using version X of file A and version Y of file B" This basically sounds like CVS.

If that is how you were using CVS, its no surprise that it gets so much hate.

OTOH, I think the common CVS workflow actually matches the modern "we don't do stable branches" workflow a lot better than git does. Basically, if you had upstream CVS branches for more than released versions of software in maintenance mode you were doing it wrong.

I also tend to yearn for the days when I didn't spend 20% of my time rebasing and merging patches together, or rewritting dozens of patches worth of git history in order to move a couple minor commit hunks between patches for some reviewer. Or just juggling 20 different -next style remote repos.

Git is one of those tools that let you endlessly play with your tools rather than getting the job done.

Re: Advantages of monolithic version control

#22

Post by joshuamorton »

shados wrote: post #18
joshuamorton wrote: post #15

Earlier quoted context omitted.

But I can say the same thing about multiple repos! I've seen more than one company now that has had the same problem: how do they patch atomic cross-repo changes onto their multiple git repos? The reasons for this can vary, but the core problem is always that. As far as I see it, there are two solutions: - Use a monorepo - Create some external database that ties multiple hashes together for use in your ecosystem. Thi…

Eventual consistency. The great thing about multi repo is the ease of decoupling the pieces so they can evolves separately (you can do that in monorepos too, but it's not quite as natural). You're free to PR changes gradually, making sure things work a couple of repos at a time, until you eventually get everything. If you can tolerate temporary inconsistencies, it allows you to scale to infinity, essentially for free…

>Eventual consistency. The great thing about multi repo is the ease of decoupling the pieces so they can evolves separately (you can do that in monorepos too, but it's not quite as natural).

I don't see how you can do this any better in a multi-repo than a monorepo though, unless you mean to the extent of simultaneously having multiple versions of the same library in your transitive deps (and thus kind of kludgily sidestepping the diamond dependency issues). Would you mind elaborating?

>You're free to PR changes gradually

This is possible in a monorepo too, by much the same means, I'd expect: you define an adaptor that you slowly migrate everyone onto, deprecate the old thing, and then optionally remove the adaptor and deprecate it too. Am I missing something?

Re: Advantages of monolithic version control

#23

Post by mnm1 »

peterevans wrote: post #13

Using multiple repositories seems like the most natural workflow to have. It's easy to make a new one; it's lightweight to do, and it allows your code base to scale naturally. You can set permissions for each repository, so if you did include some sensitive code within one repository or another, it's easy to narrow the access to them. (Yes, don't include sensitive code in a repository—but in an early-stage company, y…

With multiple repos, how do you solve the issue that the private dependencies installed by package managers are not themselves under source control when you edit them? They are in /vendor or /node_modules or whatever. It's easy to pull them down and of course, for other people's dependencies, this is fine. But say I pull out one component of my app that's used by multiple apps. I set up my private repo or maybe the package manager works straight with git. But when it pulls the dependencies down, it doesn't pull the git subdirectories and so that code is not under source control. I make edits, test, then I have to copy that code back to its original repo, resolve any conflicts, and check it in. If I need to make a change, I have to do that again. If someone worked on that code, I have to pull that and merge it in separately. Even if I manually pull it myself in the right place, I'm still dealing with a git repo inside a git repo but the two are unrelated so many tools won't work with the inner repo. Short of writing a bunch of custom scripts, is there a standard way to handle this situation that I assume anyone with multiple repos that share internal, private dependencies has?

Re: Advantages of monolithic version control

#24

Post by astral303 »

mikepurvis wrote: post #10
meddlepal wrote: post #2

I warmed up to the monorepo awhile ago but tooling and reference material that helps guide an org into adopting a monorepo and using it properly is hard to find.

People always talk about how it takes all this tooling to have an effective monorepo— it definitely does; many moons ago I was a Googler too and experienced all the fun little perforce wrappers that would check out the portions of the tree you needed to build whatever it was you were trying to build. But having things split across many repos also takes a lot of tooling too. So really, for each approach you're looking…

The difference is that multirepo large project dev and dep mgmt are problems solved today with existing, widely used tooling. With monorepo, I’m DIYing scalability onto a system not designed for it.

Re: Advantages of monolithic version control

#25

Post by peterevans »

mnm1 wrote: post #23
peterevans wrote: post #13

Using multiple repositories seems like the most natural workflow to have. It's easy to make a new one; it's lightweight to do, and it allows your code base to scale naturally. You can set permissions for each repository, so if you did include some sensitive code within one repository or another, it's easy to narrow the access to them. (Yes, don't include sensitive code in a repository—but in an early-stage company, y…

With multiple repos, how do you solve the issue that the private dependencies installed by package managers are not themselves under source control when you edit them? They are in /vendor or /node_modules or whatever. It's easy to pull them down and of course, for other people's dependencies, this is fine. But say I pull out one component of my app that's used by multiple apps. I set up my private repo or maybe the p…

Any changes you need to make to internal libraries you install via package manager would need to be made to those separate repositories that house the libraries, and then those changes would have a release made (generally this is done through a git tag, and using semantic versioning). Once you update the version, your package manager will allow you to install that update to the other places you're using it, so all you need to change is the package file.

It might seem like a pain to do it this way, especially if you're rapidly iterating on a library—it might be that your library is not really mature, or even used by more than one repository, so it may not even make sense to have that library in a separate repository to begin with! But once you do have a mature code base, semantic versioning is a really sane way of managing dependency updates for the N number of other projects which use your library.

Re: Advantages of monolithic version control

#26

Post by aarongolliver »

tzs wrote: post #19

> With a monorepo, projects can be organized and grouped together in whatever way you find to be most logically consistent, and not just because your version control system forces you to organize things in a particular way. Using a single repo also reduces overhead from managing dependencies. This is the major thing I miss about Subversion, and the fact that in Subversion a subdirectory in a repository can be checked…

I like this about perforce too. Another thing I like is that I can easily substitute subdirectories with other ones, or mask out directories I don't need. I can swap in the new code for a single module into a legacy version with a one line config change.

I also get the ability to use the depo explorer to poke around at absolutely everything that folks are working on. The search function in gitlab is basically a joke compared to this.

Unfortunately for perforce, git/gitlab seems better in basically every other way. It took me 2 years to have anything nice to say about perforce, my regular development flows we're just that painful.

Re: Advantages of monolithic version control

#27

Post by mathgladiator »

tdumitrescu wrote: post #20
amag wrote: post #6

On the other end of the spectrum, a colleague of mine recently told me about when his previous company (big, 100k+ employees) were starting to adopt git, some people seriously considered having a separate repository per file ! "Because then you can just set up your project as using version X of file A and version Y of file B" It's good we now have (at least) Google, Facebook and Microsoft as examples of companies usi…

That sweet abuse of version control reminds me of the fabled JDSL from http://thedailywtf.com/articles/the-inner-json-effect

That can't be real... is it? No. No way.

Re: Advantages of monolithic version control

#28

Post by hinkley »

tzs wrote: post #19

> With a monorepo, projects can be organized and grouped together in whatever way you find to be most logically consistent, and not just because your version control system forces you to organize things in a particular way. Using a single repo also reduces overhead from managing dependencies. This is the major thing I miss about Subversion, and the fact that in Subversion a subdirectory in a repository can be checked…

I have poked around in the git file format a tiny bit and I think the hash tree semantics aren’t incompatible with subversion’s commit tree semantics, which are what let you grab a particular sub tree cleanly.

I kinda think you might be able to convince git to let you check out a subdirectory. But I’m not sure if any of the plumbing exposes that ability or if it would take significant surgery.

Re: Advantages of monolithic version control

#29

Post by adsharma »

There is a people aspect to this trend.

There is a lot of developer movement between the companies cited and it's not surprising that people take the practices they're familiar with to their new employer.

Companies incentivize people to deprecate feature X and replace it with feature Y and celebrate the win. Much harder without a monorepo.

The counter argument is open source, where the development follows the distributed model and the difficulty of syncing the monorepo with a custom build system. Figuring out a way to leverage the QA work distro people do in coming up with a consistent cut + patches would benefit everyone regardless of repo structure.

Re: Advantages of monolithic version control

#30

Post by hinkley »

peterevans wrote: post #25
mnm1 wrote: post #23

Earlier quoted context omitted.

With multiple repos, how do you solve the issue that the private dependencies installed by package managers are not themselves under source control when you edit them? They are in /vendor or /node_modules or whatever. It's easy to pull them down and of course, for other people's dependencies, this is fine. But say I pull out one component of my app that's used by multiple apps. I set up my private repo or maybe the p…

Any changes you need to make to internal libraries you install via package manager would need to be made to those separate repositories that house the libraries, and then those changes would have a release made (generally this is done through a git tag, and using semantic versioning). Once you update the version, your package manager will allow you to install that update to the other places you're using it, so all yo…

And how do you test if a change in one of these repos fixes the problem you’re seeing? This is what we’re failing at with our multirepo.

That and resectioning code to split or combine responsibilities in different ways. Something a monorepo makes trivial.

Post reply on HN