Live data from Hacker News

Monorepo is great if you're really good

yosefk.com

81–90 of 159 posts

Re: Monorepo is great if you're really good

#81
>bad fork

How is this an argument? You merge the unforked projects and its the equivalent to multirepo. I don't follow the argument at all.

...I started writing rebuttals to the others but I guess when your argument is "yeah you can do it right but you _could_ do it wrong and I have defined the question in such a way that we err on success for multi-repo and err on failure on mono-repo" I can't really fight that.

Re: Monorepo is great if you're really good

#82
post #55

Earlier quoted context omitted.

> The theory behind the microkernel is that operating systems are complicated. Seems like a blatant straw man to me.

The advantage of microkernels is that they can be extended with “untrusted” code like hardware drivers or file systems. This runs in user space and thus any bugs in such code will not crash the kernel process. So I agree with you that Linus is presenting a straw man and your comment shouldn’t have been downvoted.

> The advantage of microkernels is that they can be extended with “untrusted” code like hardware drivers or file systems. This runs in user space and thus any bugs in such code will not crash the kernel process.

Did this advantage play out in practice? If your filesystem module goes down then every module that talks to the file system module needs to gracefully handle the failure or it will still effectively crash the system.

Or the module core dumps and the system keeps chugging on, but everything is locked up because they're waiting for the return from the crashed module. Did MINIX have a way to gracefully restart crashed modules?

Re: Monorepo is great if you're really good

#83
post #77
post #73

Earlier quoted context omitted.

Please note that Linus wrote an operating system that in practice showed greater reliability than competing commercial microkernels. I do not believe that the principles that he came to believe in that process should be dismissed as straw man arguments.

Tell that to my (lack of) graphics drivers. You can say its political but as it stands its no where near apples to apples in terms of what Windows supports vs what Linux supports.

Which video card do you have that lacks drivers for Linux? Or do you need fully open source drivers that fully support 3D acceleration and computation?

Re: Monorepo is great if you're really good

#84
post #29

Linus Torvalds said something about this, in relation to microkernels. But the gist is in how interactions between many pieces makes the whole thing complex. Here's the quote from his book "Just for Fun". "The theory behind the microkernel is that operating systems are complicated. So you try to get some of the complexity out by modularizing it a lot. The tenet of the microkernel approach is that the kernel, which is…

It boils down to gut instinct again, since there is no clear line of how much complexity to accrue before splitting something up. I think he likes to keep Linux in one repo because that makes it easier for him to watch the project and manage it. Currently, if linux gets separated into 10 components, Linus will still have to keep an eye on all 10, so from his point of view, complexity is still the same, or worse. But if he could actually let go and let someone else completely own another component, this would not be the case. Bottom line is that he is smart enough to be able to do manage a repo that large, which only proves the author's point.

Re: Monorepo is great if you're really good

#85
post #55

Earlier quoted context omitted.

The advantage of microkernels is that they can be extended with “untrusted” code like hardware drivers or file systems. This runs in user space and thus any bugs in such code will not crash the kernel process. So I agree with you that Linus is presenting a straw man and your comment shouldn’t have been downvoted.

So similar in principle to FUSE, but applied more broadly? Seems like a neat idea.

In practice it was less useful than people assumed, because:

1. Things like drivers and filesystems are usually written by a small handful of vendors, who already have rigorous engineering cultures (hardware is a lot less forgiving than say web design), and a large base of demanding users who will rapidly complain and/or sue you if you get it wrong. When was the last time you personally had a crash due to a driver or filesystem issue? It used to happen semi-frequently in the Win95 days, but there was a strong incentive for hardware manufacturers to Fix Their Shit, and so all the ones who didn't went out of business.

2. You pay a hefty performance price for that stability - since the kernel is mediating all interactions between applications and drivers, every I/O call needs to go from app to kernel to driver and back again. There's been a lot of research put into optimizing this, but the simplest solution is just don't do this, and put things like drivers & filesystems in the kernel itself.

3. The existence of Linux as a decentralized open-source operating system took away one of the main organizational reasons for microkernels. When the kernel is proprietary, then all the negotiations, documentation, & interactions needed to get code into an external codebase become quite a hassle, with everyone trying to protect their trade secrets. When it's open-source, you go look for a driver that does something similar, write yours, and then mail a patch or submit a pull request.

Re: Monorepo is great if you're really good

#86
post #29

Linus Torvalds said something about this, in relation to microkernels. But the gist is in how interactions between many pieces makes the whole thing complex. Here's the quote from his book "Just for Fun". "The theory behind the microkernel is that operating systems are complicated. So you try to get some of the complexity out by modularizing it a lot. The tenet of the microkernel approach is that the kernel, which is…

This is analogous to the debate about when to break long functions into shorter ones. The simplicity argument usually doesn't consider the increased complexity of all the interactions between the new, shorter functions. If you omit that, you get a misleading picture, or as Linus puts it, a false simplicity.

Re: Monorepo is great if you're really good

#87
post #12

I don't agree with the author that the quality of the team is what determines if a mono-repo is appropriate or not. It doesn't really matter if you use a mono-repo or not, what matters is what individual engineers are empowered to do. If engineers or even team-leads don't have permission to create a repo themselves well then you're probably going to see benefits from a monorepo. At the same time if you have say a ~10…

> Implicit in the author's reasoning is the principle that good engineers don't make mistakes; they don't break the build, not ever. They don't, ever, because the VCS refuses a push if it breaks the build. That's the problem with all those single-repo discussions. It works perfectly well if you have all the tooling that makes a single repo work like a multi-repo. And it's great because you can enforce behind the scen…

I actually tend to find the opposite - with a multi repo, CI/CD is often just plain broken and people waste significant amounts of time cargo culting working setups (and often do so badly or don't keep up with upstream changes).

Re: Monorepo is great if you're really good

#88
post #3

> the entire code base got forked, and the entire org is now held hostage by the dumbass. > Of course in a Good team, needless dependencies would be weeded out in code reviews, and a Culture would evolve over time avoiding needless dependencies. Really, the one consistent thing is that if you have a good team, you'll make it work no matter what tech or decisions you make (assuming you're also good enough to know when…

>> Of course in a Good team, needless dependencies would be weeded out in code reviews, and a Culture would evolve over time avoiding needless dependencies.

Heh. This reminded me of a different story, which I remember vaguely enough that I'll paraphrase from memory:

> "The Excel team will never go for it. Their motto is 'Find the dependencies... and eliminate them.'"

> This probably explained why the Excel team had its own C compiler.

Re: Monorepo is great if you're really good

#89
post #39
post #37

Does anybody know some monorepo horror stories? I have heard plenty of people complain about their many-repo structure and wishing for a monorepo. I would like to hear some concrete story where a monorepo went wrong. This article is just abstract opinion.

I asked a similar question a previous time the mono- vs many- repo question came out, and the few responses I got were roughly "The repo became many tens of gigabytes which was unwieldy"

Unfortunately, breaking a 10 gig repo into 5 repos often means you have to now download five 2 gig repos.

Re: Monorepo is great if you're really good

#90
post #60

Earlier quoted context omitted.

> repo A branch B can only be built with repo C branch F, but all the other repos should be master This alone is pretty much what makes me prefer monorepos. If you don't have a stable interface for all of your in-house dependencies (and nobody does early on in a project), you're doomed to spend a ton of time matching branches like this. Not to mention, a naive build process of "grab the latest everything and build it…

All modern dependency management tools solve this problem by allowing dependencies to be specified as URI's.

That comes with its own problems. You could end up spending 3 hours tracking a bug you thought you had fixed two weeks ago, only to find out that some other service is pinned to a commit three weeks ago.
Post reply on HN