Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Advantages of monolithic version control

danluu.com

Post reply on HN
Print view View original

41–50 of 144 posts

Re: Advantages of monolithic version control

#41

Post by uiri »

amag wrote: post #6

On the other end of the spectrum, a colleague of mine recently told me about when his previous company (big, 100k+ employees) were starting to adopt git, some people seriously considered having a separate repository per file ! "Because then you can just set up your project as using version X of file A and version Y of file B" It's good we now have (at least) Google, Facebook and Microsoft as examples of companies usi…

Microsoft doesn't really use a monorepo (although they may have done so in the past). Everything is moving to git, and while they've invested significant effort into getting git to handle larger repos, you don't have everything living in one repo quite the same way that Google does it.

My understanding of Google's monorepo is that search, mail, maps, etc. all live inside one repo.

Re: Advantages of monolithic version control

#42

Post by hinkley »

peterevans wrote: post #35
hinkley wrote: post #30

Earlier quoted context omitted.

And how do you test if a change in one of these repos fixes the problem you’re seeing? This is what we’re failing at with our multirepo. That and resectioning code to split or combine responsibilities in different ways. Something a monorepo makes trivial.

Ideally, your library code that you're pulling in has some unit testing to demonstrate that things are working as they should be. (If not, consider adding unit testing! It's really useful!) If that is the case, then you can isolate the likelihood of an issue as either in the library (because unit tests fail there), or in the project consuming the library (because unit tests succeed in the library). Without testing, i…

The class of problems I’m talking about are integration issues. Unit tests look good but when you put the pieces together...

When the tests that matter cross version control boundaries you pay for it. Whether the costs outweigh the benefits is something you have to think about.

Re: Advantages of monolithic version control

#43

Post by eadmund »

mnm1 wrote: post #23
peterevans wrote: post #13

Using multiple repositories seems like the most natural workflow to have. It's easy to make a new one; it's lightweight to do, and it allows your code base to scale naturally. You can set permissions for each repository, so if you did include some sensitive code within one repository or another, it's easy to narrow the access to them. (Yes, don't include sensitive code in a repository—but in an early-stage company, y…

With multiple repos, how do you solve the issue that the private dependencies installed by package managers are not themselves under source control when you edit them? They are in /vendor or /node_modules or whatever. It's easy to pull them down and of course, for other people's dependencies, this is fine. But say I pull out one component of my app that's used by multiple apps. I set up my private repo or maybe the p…

> Short of writing a bunch of custom scripts, is there a standard way to handle this situation that I assume anyone with multiple repos that share internal, private dependencies has?

You write custom scripts to reïmplement everything you'd get for free with a monorepo! Having worked at organisations with a monorepo and with many repos, I can confidently say that any team which is using multiple repos is very probably wrong — and the more repos, the more likely wrong they are. If you have more repos than team members, you are almost certainly wrong. You end up spending far more time managing cross-repo dependencies and changes than you would merging changes in a monorepo.

Multiple repos: not even once.

Re: Advantages of monolithic version control

#44

Post by jedberg »

joshuamorton wrote: post #15
jedberg wrote: post #12

The catch is the tooling. If you have the time and resources to make the tooling that is necessary to make it work specifically for your org, than great! But if you don't, then a monorepo will generally slow you down because it will require coordinating changes across a much bigger group of people. Monorepos are great for very small companies with a low communication overhead, and very large companies with the resour…

But I can say the same thing about multiple repos! I've seen more than one company now that has had the same problem: how do they patch atomic cross-repo changes onto their multiple git repos? The reasons for this can vary, but the core problem is always that. As far as I see it, there are two solutions: - Use a monorepo - Create some external database that ties multiple hashes together for use in your ecosystem. Thi…

> how do they patch atomic cross-repo changes onto their multiple git repos

If you need atomic cross repo changes, then you're doing multi-repo wrong. You need to have an upgrade path, so that you support both old and new in parallel, so you can upgrade one repo at a time.

You'd need that anyway during deployment.

Re: Advantages of monolithic version control

#45

Post by monksy »

joshuamorton wrote: post #22
shados wrote: post #18

Earlier quoted context omitted.

Eventual consistency. The great thing about multi repo is the ease of decoupling the pieces so they can evolves separately (you can do that in monorepos too, but it's not quite as natural). You're free to PR changes gradually, making sure things work a couple of repos at a time, until you eventually get everything. If you can tolerate temporary inconsistencies, it allows you to scale to infinity, essentially for free…

>Eventual consistency. The great thing about multi repo is the ease of decoupling the pieces so they can evolves separately (you can do that in monorepos too, but it's not quite as natural). I don't see how you can do this any better in a multi-repo than a monorepo though, unless you mean to the extent of simultaneously having multiple versions of the same library in your transitive deps (and thus kind of kludgily si…

The diamond problem is more due to the lack of tooling to alert you when that's going on. (Tooling on the building side and the CI/CD/Jenkins side) [Jenkins should be able to kick off other builds that depend on whats being built.

Re: Advantages of monolithic version control

#46

Post by peterevans »

hinkley wrote: post #42
peterevans wrote: post #35

Earlier quoted context omitted.

Ideally, your library code that you're pulling in has some unit testing to demonstrate that things are working as they should be. (If not, consider adding unit testing! It's really useful!) If that is the case, then you can isolate the likelihood of an issue as either in the library (because unit tests fail there), or in the project consuming the library (because unit tests succeed in the library). Without testing, i…

The class of problems I’m talking about are integration issues. Unit tests look good but when you put the pieces together... When the tests that matter cross version control boundaries you pay for it. Whether the costs outweigh the benefits is something you have to think about.

What does your test coverage look like? Perhaps you're missing something there that would have caught that bug?

Testing is, of course, no silver bullet. Tests are written by humans, and humans make mistakes—and it's pretty difficult to achieve 100% test coverage in a production system. The goal of testing is to have confidence in the code you've written.

Tests often don't need to cross version control boundaries. You can use mock data—like would be produced by the library—on the consumer side, because you can delegate responsibility for testing of that library to the library repository itself. If your tests work great with the mock data, but things are still failing, then you can infer that the mock data and the actual data are different, and your bug is in the library.

Re: Advantages of monolithic version control

#48

Post by Too »

shados wrote: post #18
joshuamorton wrote: post #15

Earlier quoted context omitted.

But I can say the same thing about multiple repos! I've seen more than one company now that has had the same problem: how do they patch atomic cross-repo changes onto their multiple git repos? The reasons for this can vary, but the core problem is always that. As far as I see it, there are two solutions: - Use a monorepo - Create some external database that ties multiple hashes together for use in your ecosystem. Thi…

Eventual consistency. The great thing about multi repo is the ease of decoupling the pieces so they can evolves separately (you can do that in monorepos too, but it's not quite as natural). You're free to PR changes gradually, making sure things work a couple of repos at a time, until you eventually get everything. If you can tolerate temporary inconsistencies, it allows you to scale to infinity, essentially for free…

Developers are way to lazy for this to work in practice. Imagine every second commit you make has to be backwards compatible, that for sure is a great way to effectively stop any refactoring of your code base, unless all your interfaces are already perfect and very fixed and you find this a feature. Or people will not care about partial cross repo dependencies resulting in randomly broken builds for others during a short time. I don't want to pull down a broken build just because some other team at that moment were only half way into pushing a feature. What should I do against that? Retry pull all repos and rebuild again?

I think the problem is that people misuse their version control as a package manager. If you want that type of behaviour, with semver and all to manage compatibility, just use a package manager to manage your dependencies, not a multirepo git contraption. You can still store your packages in one repo each if you like but at least you now get some control over interface compatibility which is a requirenent when you have multiple repos.

Re: Advantages of monolithic version control

#49

Post by hinkley »

peterevans wrote: post #46
hinkley wrote: post #42

Earlier quoted context omitted.

The class of problems I’m talking about are integration issues. Unit tests look good but when you put the pieces together... When the tests that matter cross version control boundaries you pay for it. Whether the costs outweigh the benefits is something you have to think about.

What does your test coverage look like? Perhaps you're missing something there that would have caught that bug? Testing is, of course, no silver bullet. Tests are written by humans, and humans make mistakes—and it's pretty difficult to achieve 100% test coverage in a production system. The goal of testing is to have confidence in the code you've written. Tests often don't need to cross version control boundaries. You…

For instance, I have a piece of code in a particularly gnarly modules that at this moment is disabled and has been for two sprints due to emergent behavior. First sprint it had adequate unit tests but not enough functional tests to exhibit a problem. Second sprint I fixed the testing deficiency and got the code to work end to end.

Or so I thought. The first time I turned it on it preprod I couldn't turn it back off again because some piece of data that came from five function calls away was being shared, and nobody who participated in the PR recalled that fact.

Most of the code I'm dealing with is in a single module. I have been chipping away at fixing the insane ball of mud as I can. My coworkers often aren't that lucky. They come to me for advice on how to deal with this sort of problem but crossing 2 or three modules.

There's no low-friction way for them to fix any of this. They can't just refactor because of the coordination costs, and also the loss of historical information when you move a block of code across module boundaries or try to change module boundaries. This is the prime argument for monorepos in the literature - not making irreversible decisions on Law of Demeter problems. It's not my biggest reason, but it's sufficient for most people.

Post reply on HN