Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Advantages of monolithic version control

danluu.com

Post reply on HN
Print view View original

51–60 of 144 posts

Re: Advantages of monolithic version control

#51

Post by hinkley »

DaiPlusPlus wrote: post #34
hinkley wrote: post #28

Earlier quoted context omitted.

I have poked around in the git file format a tiny bit and I think the hash tree semantics aren’t incompatible with subversion’s commit tree semantics, which are what let you grab a particular sub tree cleanly. I kinda think you might be able to convince git to let you check out a subdirectory. But I’m not sure if any of the plumbing exposes that ability or if it would take significant surgery.

Git supports this scenario as "sparse checkouts": https://git-scm.com/docs/git-read-tree - see this QA: https://stackoverflow.com/questions/4114887/is-it-possible-t...

Does this allow you to file PRs against the repository you have checked out? Or does this only work for read-only use? What about CI? How do you convince TC or Jenkins or Bamboo to do the same thing?

If this does do all that, I think this functionality needs some SEO love because this pretty much never comes up when I search for the latest ways to grab part of a repo. All I find are conversations where people are trumpeting the wrong tools for the job.

Edit: Also, this doesn't seem to let me check out a subtree the way people mean "check out a subtree". When I check out 'just foo/bar/baz' I expect to have a directory called baz as my project root. Not a directory named foo with a single grandchild named baz.

Re: Advantages of monolithic version control

#52

Post by imiric »

jedberg wrote: post #12

The catch is the tooling. If you have the time and resources to make the tooling that is necessary to make it work specifically for your org, than great! But if you don't, then a monorepo will generally slow you down because it will require coordinating changes across a much bigger group of people. Monorepos are great for very small companies with a low communication overhead, and very large companies with the resour…

I keep hearing the tooling argument in discussions about monorepos, and I'm not entirely sure what is missing in this area when compared to projects with multiple repositories.

Tools like Pants[1] or Lerna[2] solve a lot of the issues related to builds and dependency management.

What exactly are you missing _today_ that prevents your organization from adopting a monorepo approach?

[1]: https://www.pantsbuild.org/

[2]: https://lernajs.io/

Re: Advantages of monolithic version control

#53

Post by argonaut »

jedberg wrote: post #12

The catch is the tooling. If you have the time and resources to make the tooling that is necessary to make it work specifically for your org, than great! But if you don't, then a monorepo will generally slow you down because it will require coordinating changes across a much bigger group of people. Monorepos are great for very small companies with a low communication overhead, and very large companies with the resour…

Isn't the point of the article that you need less tooling with monorepos, not more (examples given: cross-repo changes, code searching, git bisect, dependency management).

Now, you might be referring to the fact that Facebook and Google have built up lots of tooling to help git/hg scale since their repos are too large and operations take too long. But that is not a problem you're going to have until you have at least a thousand engineers. At that size you need tooling for everything anyways.

Re: Advantages of monolithic version control

#55

Post by peterevans »

hinkley wrote: post #49
peterevans wrote: post #46

Earlier quoted context omitted.

What does your test coverage look like? Perhaps you're missing something there that would have caught that bug? Testing is, of course, no silver bullet. Tests are written by humans, and humans make mistakes—and it's pretty difficult to achieve 100% test coverage in a production system. The goal of testing is to have confidence in the code you've written. Tests often don't need to cross version control boundaries. You…

For instance, I have a piece of code in a particularly gnarly modules that at this moment is disabled and has been for two sprints due to emergent behavior. First sprint it had adequate unit tests but not enough functional tests to exhibit a problem. Second sprint I fixed the testing deficiency and got the code to work end to end. Or so I thought. The first time I turned it on it preprod I couldn't turn it back off a…

Yeah, I sympathize—that's a tough situation.

I think there's two ways you can look at your choice of configuration: ease of debugging, and ease of organization. When Google lays out why they use a monorepo, they are doing so because it simplifies their organization—there are no longer so many versions of so many libraries and apps they need to support; there's only one version of anything to support. Either everything works or everything fails.

But in your case, you're looking at it from the debugging point of view. It's easier to play around with the code in a monorepo. And that's totally fair point of view to have, particularly in your predicament.

That choice of a monorepo doesn't necessarily improve the quality of your code organization and interoperability. It's still going to be a bad bug to fix. It's just a little bit easier to debug.

Re: Advantages of monolithic version control

#56

Post by gmueckl »

hyperpape wrote: post #37
StillBored wrote: post #21

Earlier quoted context omitted.

If that is how you were using CVS, its no surprise that it gets so much hate. OTOH, I think the common CVS workflow actually matches the modern "we don't do stable branches" workflow a lot better than git does. Basically, if you had upstream CVS branches for more than released versions of software in maintenance mode you were doing it wrong. I also tend to yearn for the days when I didn't spend 20% of my time rebasin…

What workflow do you use with git that has you doing so much juggling?

Pick one. I have yet to encounter a workflow that did not introduce tons of extra fiddling and stupid busywork.

Re: Advantages of monolithic version control

#58

Post by draw_down »

peterevans wrote: post #13

Using multiple repositories seems like the most natural workflow to have. It's easy to make a new one; it's lightweight to do, and it allows your code base to scale naturally. You can set permissions for each repository, so if you did include some sensitive code within one repository or another, it's easy to narrow the access to them. (Yes, don't include sensitive code in a repository—but in an early-stage company, y…

You only create the repo once, whether that's easy to do is beside the point. Sure it's easy. The problem is deploying related changes together across repos. Every way of doing that sucks.

Re: Advantages of monolithic version control

#59

Post by idunno246 »

hinkley wrote: post #30
peterevans wrote: post #25

Earlier quoted context omitted.

Any changes you need to make to internal libraries you install via package manager would need to be made to those separate repositories that house the libraries, and then those changes would have a release made (generally this is done through a git tag, and using semantic versioning). Once you update the version, your package manager will allow you to install that update to the other places you're using it, so all yo…

And how do you test if a change in one of these repos fixes the problem you’re seeing? This is what we’re failing at with our multirepo. That and resectioning code to split or combine responsibilities in different ways. Something a monorepo makes trivial.

back in the day, eclipse would handle this fine - you normally have the jars, but if you need to edit the underlying project you can tell it that jar is this folder on disk(which is a different repo), and it loads it correctly as another project in the workspace replacing the jar.

Re: Advantages of monolithic version control

#60

Post by imiric »

shados wrote: post #18
joshuamorton wrote: post #15

Earlier quoted context omitted.

But I can say the same thing about multiple repos! I've seen more than one company now that has had the same problem: how do they patch atomic cross-repo changes onto their multiple git repos? The reasons for this can vary, but the core problem is always that. As far as I see it, there are two solutions: - Use a monorepo - Create some external database that ties multiple hashes together for use in your ecosystem. Thi…

Eventual consistency. The great thing about multi repo is the ease of decoupling the pieces so they can evolves separately (you can do that in monorepos too, but it's not quite as natural). You're free to PR changes gradually, making sure things work a couple of repos at a time, until you eventually get everything. If you can tolerate temporary inconsistencies, it allows you to scale to infinity, essentially for free…

This is great in theory, but it requires _a lot_ of discipline and responsibility around dependency management from the developers.

What ends up happening in practice is that all these seemingly independent components have very strict—and sometimes even unspecified—dependencies between each other.

People create "base" packages that all other packages have a strict or loose dependency on, so when that package changes, it's a guessing game if it introduced breaking changes downstream.

With a monorepo, these dependencies are tracked and always visible, and integration testing between all dependent components becomes much easier.

It's very unlikely that you'll find truly decoupled code bases within an organization. It goes against the point of grouping people to work on a common goal to begin with.

Post reply on HN