Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Monorepo is great if you're really good

yosefk.com

Post reply on HN
Print view View original

71–80 of 159 posts

Re: Monorepo is great if you're really good

#71

Post by jayd16 »

bartread wrote: post #30
jasim wrote: post #29

Linus Torvalds said something about this, in relation to microkernels. But the gist is in how interactions between many pieces makes the whole thing complex. Here's the quote from his book "Just for Fun". "The theory behind the microkernel is that operating systems are complicated. So you try to get some of the complexity out by modularizing it a lot. The tenet of the microkernel approach is that the kernel, which is…

> "...The simplicity you try to reach is a false simplicity." Also applies to some microservice architectures I've seen. People completely disregard the complexity (and overhead!) of the interactions between microservices.

This is a poor take away. Even in a monolith, you still want to separate your concerns, yes? Your code should be almost as abstracted in a monolith as they are in micro services. In microservices the code is just deployed across multiple instances.

RPCs and local method calls both need to be fault tolerant and race condition free. As you break up datastores, transactions become more complex but certainly you had a specific reason to do that so the complexity isn't a choice.

Sure the communication layer is added complexity but that too is should be abstracted into boilerplate such that you shouldn't have to think about it. Overall the added complexity requires more work but it shouldn't really make your business logic problems more complicated.

Re: Monorepo is great if you're really good

#72

Post by tiglionabbit »

madhadron wrote: post #50
JamesBarney wrote: post #13

Ugh I don't like his use of the word dumbass. Coding is hard, it's easy to break things. We've developed lots of tools of strategies so we don't have to rely on people getting it right every time. Branching: getting forked by your worst programmer This example seems contrived I've never worked anywhere where having a fork that works for some scenarios and not others is tolerated for long. This would be given the high…

> There are few organizations that have so much code they break available source control solutions but simultaneously don't have the technical expertise to manage a monorepo that large. Facebook's Mononoke ( https://github.com/facebookexperimental/mononoke ) pretty much removes that argument. They outgrew their current source control, and that's their path forward for the next couple orders of magnitude.

Not sure what you're implying about mononoke but the readme says:

> The version that we provide on GitHub does not build yet.

So, maybe eventually. They found that Git didn't do well at their scale, so they modified mercurial instead.

I'm all in favor of a company-wide mono-repo if it doesn't have scaling issues.

Re: Monorepo is great if you're really good

#73

Post by btilly »

sorbits wrote: post #55
SomeOldThrow wrote: post #47

Earlier quoted context omitted.

> The theory behind the microkernel is that operating systems are complicated. Seems like a blatant straw man to me.

The advantage of microkernels is that they can be extended with “untrusted” code like hardware drivers or file systems. This runs in user space and thus any bugs in such code will not crash the kernel process. So I agree with you that Linus is presenting a straw man and your comment shouldn’t have been downvoted.

Please note that Linus wrote an operating system that in practice showed greater reliability than competing commercial microkernels. I do not believe that the principles that he came to believe in that process should be dismissed as straw man arguments.

Re: Monorepo is great if you're really good

#74

Post by truncate »

jasim wrote: post #29

Linus Torvalds said something about this, in relation to microkernels. But the gist is in how interactions between many pieces makes the whole thing complex. Here's the quote from his book "Just for Fun". "The theory behind the microkernel is that operating systems are complicated. So you try to get some of the complexity out by modularizing it a lot. The tenet of the microkernel approach is that the kernel, which is…

If I'm not mistaken, another argument for microkernel was the isolation of each modules. For example, if I'm using some driver X and it crashes, rest of my system will continue to work fine. That's not the case with monolithic kernel. I think this safety is pretty cool to have.

Although, Linux still probably does the best job of being stable compared to rest of OS I use (Windows, macOS). I can't recall the last time I got into kernel panic or crashed (despite worse drivers in some cases).

Re: Monorepo is great if you're really good

#75

Post by humanrebar »

DanFeldman wrote: post #28

My team just switched to a monorepo. It's been only a few weeks, so I can't claim any results yet, but we've lived w/ the pain of poly-repo for long enough that we were ready to invest in a single repo. We've spent a lot of time building and iterating a unified ci/cd environment to support the new repo. Previously each project had it's own test/deploy/build/publish story and usually it's own jenkins project. Now, eac…

I suspect for that number of projects monorepos make a lot of sense.

The major technology organizations we hear about usually have at least several monorepos, due to the legacies of acquisitions and mergers if nothing else.

At the scale of thousands of subprojects, I am not entirely sure the benefits are as advertised. There will be support of subprojects forked to public github.com or gitlab.com if nothing else. And there will be external dependencies to manage; system level libraries like openssl and libc if nothing else. Even if they are vendored in to the monorepo, any upstream regression is a significant problem in a monorepo... and the problem sometimes has to be solved in a big bang instead of incrementally.

Re: Monorepo is great if you're really good

#76

Post by detaro »

truncate wrote: post #74
jasim wrote: post #29

Linus Torvalds said something about this, in relation to microkernels. But the gist is in how interactions between many pieces makes the whole thing complex. Here's the quote from his book "Just for Fun". "The theory behind the microkernel is that operating systems are complicated. So you try to get some of the complexity out by modularizing it a lot. The tenet of the microkernel approach is that the kernel, which is…

If I'm not mistaken, another argument for microkernel was the isolation of each modules. For example, if I'm using some driver X and it crashes, rest of my system will continue to work fine. That's not the case with monolithic kernel. I think this safety is pretty cool to have. Although, Linux still probably does the best job of being stable compared to rest of OS I use (Windows, macOS). I can't recall the last time…

I've been positively suprised by Windows in the presence of some issues with bad graphics drivers. On Windows, the screen flickers, the "guilty" app dies and a popup appears in the corner "sorry, the graphics subsystem had to be restarted". Whereas GPU driver issues on Linux typically leave you at the text console at best. (a driver going totally haywire can of course bring down both entirely)

Re: Monorepo is great if you're really good

#77

Post by jayd16 »

btilly wrote: post #73
sorbits wrote: post #55

Earlier quoted context omitted.

The advantage of microkernels is that they can be extended with “untrusted” code like hardware drivers or file systems. This runs in user space and thus any bugs in such code will not crash the kernel process. So I agree with you that Linus is presenting a straw man and your comment shouldn’t have been downvoted.

Please note that Linus wrote an operating system that in practice showed greater reliability than competing commercial microkernels. I do not believe that the principles that he came to believe in that process should be dismissed as straw man arguments.

Tell that to my (lack of) graphics drivers. You can say its political but as it stands its no where near apples to apples in terms of what Windows supports vs what Linux supports.

Re: Monorepo is great if you're really good

#78

Post by andrewprock »

sandGorgon wrote: post #21

>>>* They don't like to have to clone lots of other repos, and to then worry about their versions (in part because the tooling support for this might be less than great).* I think this is the argument around which the whole post is made. Everyone does want to work in a small space where they control everything. I want to see git log with just my code commits - so I'll make a microservice out of it. All other argument…

It's not that hard to make a library/module, and then type:

    git log .

Re: Monorepo is great if you're really good

#79

Post by jnurmine »

Is the monorepo/multirepo choice really the most important thing to consider?

Branching: monorepo or not, if a feature-incomplete development branch for one of the supported targets can "hold the entire organization as a hostage" then the SCM people, and/or persons responsible of the SCM policy, should do some introspection...

Why are deliveries done from a branch which is obviously still in development? Why does code-to-be-released need to depend on incomplete work? Why aren't something like "topic branches" used?

Modularity: monorepo or not, problems will certainly appear when the complexity of implementation outpaces the capacity created by the design. To get modularity, one needs actual modules with properly designed (=not brittle, DRY, KISS, YAGNI, SOLID, etc. etc.) interfaces between the modules. Now, does monorepo/multirepo really play a role here at all? If everyday changes are constantly modifying the module interfaces in incompatible ways which breaks existing code, this speaks something about the design, or rather the insufficiency of it.

Of course, every project and team is different. However, even if a locally optimal choice for the monorepo vs. multirepo question is found, problems existing regardless of monorepo/multirepo will still be there.

Re: Monorepo is great if you're really good

#80

Post by efaref »

detaro wrote: post #76
truncate wrote: post #74

Earlier quoted context omitted.

If I'm not mistaken, another argument for microkernel was the isolation of each modules. For example, if I'm using some driver X and it crashes, rest of my system will continue to work fine. That's not the case with monolithic kernel. I think this safety is pretty cool to have. Although, Linux still probably does the best job of being stable compared to rest of OS I use (Windows, macOS). I can't recall the last time…

I've been positively suprised by Windows in the presence of some issues with bad graphics drivers. On Windows, the screen flickers, the "guilty" app dies and a popup appears in the corner "sorry, the graphics subsystem had to be restarted". Whereas GPU driver issues on Linux typically leave you at the text console at best. (a driver going totally haywire can of course bring down both entirely)

This is mostly because of a long history of crummy graphics drivers on Windows led to countless bluescreens. In the past, if your Windows machine bluescreened, it was a safe bet that the graphics driver was the cause.

Microsoft had enough telemetry telling them this that they spent a large effort restructuring the graphics driver subsystem so that it could crash and burn and be restarted without affecting the rest of the system.

Although Linux already has the isolation, it doesn't have the clean recovery. Since the year of the Linux desktop hasn't arrived yet, Linux is yet to make this journey.

Post reply on HN