Live data from Hacker News

Advantages of monolithic version control

danluu.com

91–100 of 144 posts

Re: Advantages of monolithic version control

#91
Here is a naive question:

Let's say I want a small application with flask and angular.

I create a single repo for both flask and angular. I put everything flask in one sub folder called backbend and I put everything angular in another folder called frontend

WIP here: https://github.com/kusl/flaskexperiment or https://git.sr.ht/%7Ekus/flaskexperiment/

Now the problems are just starting: how do I set up ci for all my projects? Travis ci expects a single file at the root of the project and so does gitlab ci (hi sid, big fan)

I am sad I can't talk to the experts at Google about how they navigated these problems. I understand Google has many enemies that are constantly trying to exploit whatever but I still wish we could have a more open conversation here.

Re: Advantages of monolithic version control

#93

How do people handle the case where in a large repository, with people committing almost constantly, pushing your changes to a git server on the other side of the world becomes quite tricky. By the time my push gets to the server it seems someone has gotten in before me and my repo is out of date. I have to pull the new changes and try again. During busy parts of the day (my afternoon, the U.S morning) you might have…

You don't ever push straight to the repo. You enqueue your changeset to be pushed by a central system. At Facebook we call this "asynchronous landing" and you're right, before this became a thing about five years back, at certain times of day it was very tricky to push your changes out. Once you switch to this model, you can do convenient things like landing straight from the review system (by a "Ship it" button), la…

This does sound like the answer. I wonder if our Atlassian solution supports this, perhaps they call it by another name.

Re: Advantages of monolithic version control

#94
post #75

Earlier quoted context omitted.

That seems very weird. Do you happen to know the reasoning behind it?

Arch maintainer here. Arch Linux doesn't use git, it uses svn still. Each arch package-repository (core, extra, community...) is a single SVN repository. Packages are split in subdirectories in that repository and checked out individually. Every maintainer deals with a few dozen packages individually, not the whole. There are seldom any commits that span multiple repositories. What you were linked is the "svn to git"…

> Having a monorepo split up in lots of microrepos is straight up not possible in git.

Not saying you should, but if your goal is to have a single repo (VCS) per repo (Arch) within which all packages are stored, and replicate the {core/{foo,bar,baz},extra/{qux,tor,meh}} tree, there could be many ways to do it depending on the exact needs, by leveraging GIT_DIR, GIT_WORK_TREE, GIT_OBJECT_DIRECTORY, GIT_ALTERNATE_OBJECT_DIRECTORIES, clone --single-branch and checkout --orphan.

Single monorepo with orphan branches, single clone, multiple work trees:

    # create monorepo
    mkdir core
    cd core
    git init --bare .git
    export GIT_DIR=$(PWD)/.git
    # add package in its own branch isolated
    export GIT_WORK_TREE=$(PWD)/bash
    mkdir bash
    cd bash
    git checkout --orphan bash
    git add PKGBUILD
    git commit
    # switch package
    export GIT_WORK_TREE=$(PWD)/readline
    mkdir readline
    cd readline
    git checkout --orphan readline
    git add PKGBUILD
    git commit
    # back to bash
    export GIT_WORK_TREE=$(PWD)/bash
    cd bash
    git checkout bash 
But each time you switch you have to switch the branch too since HEAD is in core/.git, so to work around that you can share the object dir.

Single monorepo with orphan branches, multiple clones but shared object dir, multiple work trees:

    mkdir core
    cd core
    export GIT_OBJECT_DIRECTORY=$(PWD)/.git_objects
    mkdir bash
    cd bash
    GIT_DIR=$(PWD)/.git git init # otherwise .git will be created next to .git_objects
    git checkout --orphan bash
    touch PKGBUILD
    git add PKGBUILD
    git commit -m "Add package: bash"
    cd ..
    mkdir readline
    cd readline
    GIT_DIR=$(PWD)/.git git init
    git checkout --orphan readline
    touch PKGBUILD
    git add PKGBUILD
    git commit -m "Add package: readline"
    # back to bash
    cd ../bash
    git branch # just to check it's "bash", not "readline"
    # clone existing package
    cd ..
    git clone some.where:core.git --single-branch --branch libarchive
    cd libarchive
That's from the top of my head, and well it very much depends on what you want to achieve as a workflow.

Re: Advantages of monolithic version control

#95

Here is a naive question: Let's say I want a small application with flask and angular. I create a single repo for both flask and angular. I put everything flask in one sub folder called backbend and I put everything angular in another folder called frontend WIP here: https://github.com/kusl/flaskexperiment or https://git.sr.ht/%7Ekus/flaskexperiment/ Now the problems are just starting: how do I set up ci for all my p…

> Travis ci expects a single file at the root of the project and so does gitlab ci

GitLab has a related issue to support "Several .gitlab-ci.yml for monorepos" (https://gitlab.com/gitlab-org/gitlab-ce/issues/18157).

Re: Advantages of monolithic version control

#96
Is there any Free Software VCS that specifically targets monorepos?

If not, which of the "usual" VCS are best suited for monorepos? CVS? Subversion? Darcs? Bazaar? Mercurial? Git?

Should one use Mercurial simply because Facebook uses (and patches) it, or are the better choices for small-to-midsize organizations?

Re: Advantages of monolithic version control

#97
There should be " (2015)" added to the title.

(BTW, I was slightly confused not to find a single date on this blog post, and the HTTP headers were also useless. But at least there are rough timestamps at the main site.)

Re: Advantages of monolithic version control

#98
post #3

I initially found the idea of monolithic repositories hard to digest. But now I think it's a good idea for some of the reasons outlined in the article. Namely, it's very easy to depend on other code that the organization has created. In the open source world, I have found some Unix distros use the same model. I know it's not as extreme, but the principle is quite similar. For example, in Nixpkgs all package definitio…

I initially found the idea of monolithic repositories hard to digest. But now I think it's a good idea for some of the reasons outlined in the article. Namely, it's very easy to depend on other code that the organization has created. I remember the days when monorepo was the norm, and distributed version control was the weird, kooky idea. Mainstream programmers had knee-jerk notions that all managed environments were…

[deleted]

Re: Advantages of monolithic version control

#99

Earlier quoted context omitted.

> Namely, it's very easy to depend on other code that the organization has created. On the other hand, I've seen the other sides of this in monorepos: 1. It's too easy to depend on code, so there is dependency bloat when something simpler would work just as well. 2. It's relatively hard to depend on things not in the repo, reinforcing not-invented-here culture.

2 is simple. Import everything you need to depend on into the repo. Google has 3rdparty directory in its mono repo to put them.

That's an approach. Don't know if I'd call it simple. Your monorepo would start looking like an artifact repository at some point, with multiple versions of products.

And Google's approach, I'm sure, requires a bit of standardization and tooling investment. It's not clear to me that equivalent conformance and investment in monorepos and a package manager wouldn't work just as well.

Re: Advantages of monolithic version control

#100

I didn't notice a link to "Software Engineering at Google" which is a great article that goes into the monorepo argument as well as a lot of other cool practices. https://arxiv.org/abs/1702.01715

Even more relevant IMHO is this: Why Google Stores Billions of Lines of Code in a Single Repository

https://cacm.acm.org/magazines/2016/7/204032-why-google-stor...

Post reply on HN