Doesn't history become a mess with Monorepos?
Alternatively, it is a mess to track connected changes through the history of many repos. In this case, I'd think default tooling is actually better to solve the problem that the mono-repo scenario has.
Advantages of monolithic version control
101–110 of 144 posts
Re: Advantages of monolithic version control
#102Monorepos are great if you are a monoculture. I don't find it surprising that Google, Facebook and Twitter all enjoy and benefit from monorepos. I also don't find life inside the monochromatic empire particularly appealing.
I think the benefits that massive corporations derive from monorepos demonstrates how massive corporations are a net negative to society. Imagine if, instead of an increasingly centralised and closed culture in technology, companies were small enough and interdependent enough that a decentralised model became a net positive.
Re: Advantages of monolithic version control
#103Earlier quoted context omitted.
Arch maintainer here. Arch Linux doesn't use git, it uses svn still. Each arch package-repository (core, extra, community...) is a single SVN repository. Packages are split in subdirectories in that repository and checked out individually. Every maintainer deals with a few dozen packages individually, not the whole. There are seldom any commits that span multiple repositories. What you were linked is the "svn to git"…
> Having a monorepo split up in lots of microrepos is straight up not possible in git. Not saying you should, but if your goal is to have a single repo (VCS) per repo (Arch) within which all packages are stored, and replicate the {core/{foo,bar,baz},extra/{qux,tor,meh}} tree, there could be many ways to do it depending on the exact needs, by leveraging GIT_DIR, GIT_WORK_TREE, GIT_OBJECT_DIRECTORY, GIT_ALTERNATE_OBJEC…
Orphan branches removes the (seldom-used, but still used) ability to commit on multiple microrepos at once.
Re: Advantages of monolithic version control
#104Re: Advantages of monolithic version control
#105The catch is the tooling. If you have the time and resources to make the tooling that is necessary to make it work specifically for your org, than great! But if you don't, then a monorepo will generally slow you down because it will require coordinating changes across a much bigger group of people. Monorepos are great for very small companies with a low communication overhead, and very large companies with the resour…
But I can say the same thing about multiple repos! I've seen more than one company now that has had the same problem: how do they patch atomic cross-repo changes onto their multiple git repos? The reasons for this can vary, but the core problem is always that. As far as I see it, there are two solutions: - Use a monorepo - Create some external database that ties multiple hashes together for use in your ecosystem. Thi…
> Create some external database that ties multiple hashes together for use in your ecosystem. This also requires re-inventing bisect on top of this database. I'm sure the intelligent people of HN can come up with the multitude of other tools you need to modify to make this work, but its not trivial.
The thing is, there are already many mature well understood tools that you're probably already using in your organization whose goal is to tie multiple hashes together. Back in the day they were called package managers and had names like dpkg and rpm, and we'd use something like "pkg-config" to link to a specific one. These days we have docker and nix and a hundred language-specific dependency managers to solve every little variation on "tie these specific hashes together".
Monorepos still require tooling to manage effectively, but that tooling is at a disadvantage because it's not already being ubiquitously used. And unless you're bringing your entire dependency tree, recursively, all the way down to your OS, into the monorepo then you're still going to have to be dealing with those external dependencies one way or another.
And if you're not versioning the world, then it's really just an argument of the appropriate size of a unit of functionality in a multi-repo, and in that case "small enough to be well supported by existing tooling" isn't a terrible upper-bounds to pick for most people.
Re: Advantages of monolithic version control
#106Earlier quoted context omitted.
What workflow do you use with git that has you doing so much juggling?
Pick one. I have yet to encounter a workflow that did not introduce tons of extra fiddling and stupid busywork.
http://scottchacon.com/2011/08/31/github-flow.html is the style I favour. It certainly involves steps but every distinction it forces you to draw feels meaningful (at least to me), with the possible exception of the commit/push distinction which is a performance optimisation (I certainly wouldn't want to have to wait for a push on every commit).
Re: Advantages of monolithic version control
#107Earlier quoted context omitted.
> Having a monorepo split up in lots of microrepos is straight up not possible in git. Not saying you should, but if your goal is to have a single repo (VCS) per repo (Arch) within which all packages are stored, and replicate the {core/{foo,bar,baz},extra/{qux,tor,meh}} tree, there could be many ways to do it depending on the exact needs, by leveraging GIT_DIR, GIT_WORK_TREE, GIT_OBJECT_DIRECTORY, GIT_ALTERNATE_OBJEC…
Orphan branches removes the (seldom-used, but still used) ability to commit on multiple microrepos at once.
Haha definitely!
That's a trade-off I'd be willing to make though given other advantages git offers. I wonder how mercurial would fare on that regard (possibly enhanced with a bespoke extension). Not that there's any reason for your tooling to change since it fits the bill so well :)
Anyway for archmac I went the boring way and stuffed everything as subdirs inside a single monorepo as I reckoned it'd just be easier for contributions. Definitely not the same scope as Arch Linux though.
Re: Advantages of monolithic version control
#108Earlier quoted context omitted.
People always talk about how it takes all this tooling to have an effective monorepo— it definitely does; many moons ago I was a Googler too and experienced all the fun little perforce wrappers that would check out the portions of the tree you needed to build whatever it was you were trying to build. But having things split across many repos also takes a lot of tooling too. So really, for each approach you're looking…
The difference is that multirepo large project dev and dep mgmt are problems solved today with existing, widely used tooling. With monorepo, I’m DIYing scalability onto a system not designed for it.
Oh, I totally agree with you. I do robotics, and my company inherited a multi-repo approach kind of by default from our upstream ecosystem (ROS). So we get a lot of the benefit of our upstream having tackled a number of the issues around having a product made up of hundreds of repos.
Now, our upstream doesn't _actually_ ship very much, and what they do ship is relatively slow moving. So we've had to extend the supplied tooling in various ways to truly meet our needs.
Re: Advantages of monolithic version control
#109Earlier quoted context omitted.
Orphan branches removes the (seldom-used, but still used) ability to commit on multiple microrepos at once.
Haha definitely! That's a trade-off I'd be willing to make though given other advantages git offers. I wonder how mercurial would fare on that regard (possibly enhanced with a bespoke extension). Not that there's any reason for your tooling to change since it fits the bill so well :) Anyway for archmac I went the boring way and stuffed everything as subdirs inside a single monorepo as I reckoned it'd just be easier f…
FWIW there is a move towards git in Arch, which is happening slowly and I'm not sure what it'll end up looking like (probably 1 repo per pkgbuild on a custom git server, just like the AUR works right now). I was mostly talking about why svntogit looks the way it does :)
I often think about this issue of monorepos... git is such a wonderful tool, but the problems of subrepos and monorepos comes up so often it's surprising there isn't a definitive answer in git.
Re: Advantages of monolithic version control
#110Earlier quoted context omitted.
What does your test coverage look like? Perhaps you're missing something there that would have caught that bug? Testing is, of course, no silver bullet. Tests are written by humans, and humans make mistakes—and it's pretty difficult to achieve 100% test coverage in a production system. The goal of testing is to have confidence in the code you've written. Tests often don't need to cross version control boundaries. You…
>Tests often don't need to cross version control boundaries. You can use mock data—like would be produced by the library—on the consumer side, because you can delegate responsibility for testing of that library to the library repository itself. I take it as a rule of thumb that a more realistic test is better than a less realistic test. I can't think of any reason why you would want to mock anything if using the real…
Indeed, plus using a mock adds a whole new class of bugs, arising from the mock not precisely mimicking the real component or subsystem. As the sorts of bugs you are trying flush out in integration testing generally result from unforeseen interactions between components, it is more likely than chance that the mock will not behave in a way that will reveal the presence of the error.