Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Monorepo is great if you're really good

yosefk.com

Post reply on HN
Print view View original

141–150 of 159 posts

Re: Monorepo is great if you're really good

#141

Post by SomeOldThrow »

btilly wrote: post #73
sorbits wrote: post #55

Earlier quoted context omitted.

The advantage of microkernels is that they can be extended with “untrusted” code like hardware drivers or file systems. This runs in user space and thus any bugs in such code will not crash the kernel process. So I agree with you that Linus is presenting a straw man and your comment shouldn’t have been downvoted.

Please note that Linus wrote an operating system that in practice showed greater reliability than competing commercial microkernels. I do not believe that the principles that he came to believe in that process should be dismissed as straw man arguments.

Source? Speed I can imagine, but not reliability.

Re: Monorepo is great if you're really good

#142

Post by Too »

joshschreuder wrote: post #108
DanFeldman wrote: post #28

My team just switched to a monorepo. It's been only a few weeks, so I can't claim any results yet, but we've lived w/ the pain of poly-repo for long enough that we were ready to invest in a single repo. We've spent a lot of time building and iterating a unified ci/cd environment to support the new repo. Previously each project had it's own test/deploy/build/publish story and usually it's own jenkins project. Now, eac…

How do you handle building changes to just one of those projects? Can Jenkins do that (easily)? I think that's the big thing that always puts me off monorepo... We'd basically be going from ten 5 minute builds to one 50 minute build if it wasn't possible to do incremental builds. IIRC Google and MS have purpose built tools that do impact detection to work out what to build for their monorepos to keep build times down…

If you have to do make clean in a monorepo you are pretty much toast. Tooling for impact detection and reliable makefiles that always succeed incremental builds is absolutely crucial.

In a way this is one of the hallmarks of a monorepo - Interfaces and dependencies changing so quickly it becomes too troublesome for humans to categorize (and re-categorize) them into repositories, so you let a machine (makefiles) do the work instead. And even without a monorepo you still have the same problem, eventually you will have to integrate all your mini repos into one final product, which you want to have tested. This is something you want to do as frequently as possible, ideally on every commit, not by doing major version-steps of sub-projects.

Re: Monorepo is great if you're really good

#143

Post by [deleted] »

nostrademons wrote: post #85
liability wrote: post #61

Earlier quoted context omitted.

So similar in principle to FUSE, but applied more broadly? Seems like a neat idea.

In practice it was less useful than people assumed, because: 1. Things like drivers and filesystems are usually written by a small handful of vendors, who already have rigorous engineering cultures (hardware is a lot less forgiving than say web design), and a large base of demanding users who will rapidly complain and/or sue you if you get it wrong. When was the last time you personally had a crash due to a driver or…

[deleted]

Re: Monorepo is great if you're really good

#144

Post by Too »

sagarm wrote: post #118
kevan wrote: post #111

Earlier quoted context omitted.

If you're doing a monorepo I think it's strongly implied that you'll also use a build system (Blaze/Brazil/BuildXL etc) that has granular compilation units and output caching so build time doesn't scale linearly with the company's total codebase. It's definitely important to consider before jumping in. Going from 5m to 50m compile times would be a major issue for me.

Err what is even the alternative? Even a makefile would provide incremental builds.

A good makefile provides incremental builds in every possible scenario of source file changes. It's very easy to write bad makefiles that don't catch modified or removed header files, or even worse when code generation or other complicated build logic is involved. Because a lot of developers seem to think "random build fail? oh just do make clean" is an acceptable workaround.

Re: Monorepo is great if you're really good

#145

Post by Too »

flukus wrote: post #130
qznc wrote: post #37

Does anybody know some monorepo horror stories? I have heard plenty of people complain about their many-repo structure and wishing for a monorepo. I would like to hear some concrete story where a monorepo went wrong. This article is just abstract opinion.

Not horror stories, but I've used mono repos in a company with 50+ projects and the results tend to be tight coupling, macro level spaghetti and libraries being left to atrophy due to fear of change. For example I can be working on project B and need to make a change to Lib A, so I make the change commit my work and now project Z broke. Now I have to learn whatever the hell project Z is because it's not my responsibi…

Sounds like a feature and not a bug. It prevents irresponsible breaking changes of lib A interface and just hoping that some other Z team will clean up after you.

Postponing the required Z change to later could be seen as beneficial in some scenarios but what if the change you made to lib A was a security fix, then you would want all apps of that lib to be forced updated right away. Then your your change should be backwards compatible, monorepo or not.

If you want to have reusable components then make sure they are reusable, if you want a special version of lib A that only works with lib B you are essentially forking lib A making it not longer a reusable lib, just a subdir for project B. Interface versioning could help with such non backwards compatible changes, in a monorepo you normally do this with a /2.0-directory.

Re: Monorepo is great if you're really good

#147

Post by jandrese »

sorbits wrote: post #138
jandrese wrote: post #82

Earlier quoted context omitted.

> The advantage of microkernels is that they can be extended with “untrusted” code like hardware drivers or file systems. This runs in user space and thus any bugs in such code will not crash the kernel process. Did this advantage play out in practice? If your filesystem module goes down then every module that talks to the file system module needs to gracefully handle the failure or it will still effectively crash th…

> Did this advantage play out in practice? If your filesystem module goes down then every module that talks to the file system module needs to gracefully handle the failure or it will still effectively crash the system. If the file system process crashes then in theory the OS would simply relaunch it. But your core services should be stable, it’s more about extensions, for example you may want to have virtual file sy…

That's the theory yes, but I was asking about real life. Did those early microkernel systems actually deliver?

Also, for anything stateful, like a filesystem, simply relaunching it may not be sufficient. You need to make sure it hasn't lost any data in the crash and possibly rewind some state changes in related modules.

Re: Monorepo is great if you're really good

#148

Post by humanrebar »

School-Cotton wrote: post #127
humanrebar wrote: post #75

Earlier quoted context omitted.

I suspect for that number of projects monorepos make a lot of sense. The major technology organizations we hear about usually have at least several monorepos, due to the legacies of acquisitions and mergers if nothing else. At the scale of thousands of subprojects, I am not entirely sure the benefits are as advertised. There will be support of subprojects forked to public github.com or gitlab.com if nothing else. And…

FWIW at least 95% (anecdotally) of Facebook’s main code is in two gigantic monorepos: fbsource and www. (The other major repos are for configuration-related stuff). Last I heard there were plans to move www into fbsource. There are certainly not random dependencies on public GitHub pages. Everything is versioned. There is a mind boggling amount of custom tooling to make this work.

The versioned external dependencies work for systems that support semantic versioning. Some dynamic languages. Some C. Definitely nothing with a non-C ABI.

But "not working" looks like fixing an unknown number of bugs across the various subrepos. Because permanently forking upstream it never applying security patches isn't a good business model.

Re: Monorepo is great if you're really good

#149

Post by DanFeldman »

joshschreuder wrote: post #108
DanFeldman wrote: post #28

My team just switched to a monorepo. It's been only a few weeks, so I can't claim any results yet, but we've lived w/ the pain of poly-repo for long enough that we were ready to invest in a single repo. We've spent a lot of time building and iterating a unified ci/cd environment to support the new repo. Previously each project had it's own test/deploy/build/publish story and usually it's own jenkins project. Now, eac…

How do you handle building changes to just one of those projects? Can Jenkins do that (easily)? I think that's the big thing that always puts me off monorepo... We'd basically be going from ten 5 minute builds to one 50 minute build if it wasn't possible to do incremental builds. IIRC Google and MS have purpose built tools that do impact detection to work out what to build for their monorepos to keep build times down…

It was a bit hacky, but we've basically implemented some of the stuff in [1] to achieve incremental builds. If a pr changes projects a,b,c and not x,y,z then it will only build a,b,c. But it's not truly incremental right now, as it won't test things that depend on A/B/C.

We have plans to use Bazel in the future, but you have to boil the ocean when moving to bazel and get everything ever inside bazel before you get any benefit out of it.

Jenkins can't do it "easily" but it definitely can. I'd be happy to share our Jenkinsfile if you'd like.

Our finding of changes is something like:

#!/bin/bash set -euxo pipefail

COMPARE_BRANCH=$1

MERGE_BASE=`git merge-base $COMPARE_BRANCH HEAD` FILES_CHANGED=$(git diff --name-only $MERGE_BASE | grep '/') echo ${FILES_CHANGED} | xargs dirname | cut -d "/" -f 1 | sort | uniq

[1] blog.shippable.com/ci/cd-of-microservices-using-mono-repos

Re: Monorepo is great if you're really good

#150

Post by sorbits »

jandrese wrote: post #147
sorbits wrote: post #138

Earlier quoted context omitted.

> Did this advantage play out in practice? If your filesystem module goes down then every module that talks to the file system module needs to gracefully handle the failure or it will still effectively crash the system. If the file system process crashes then in theory the OS would simply relaunch it. But your core services should be stable, it’s more about extensions, for example you may want to have virtual file sy…

That's the theory yes, but I was asking about real life. Did those early microkernel systems actually deliver? Also, for anything stateful, like a filesystem, simply relaunching it may not be sufficient. You need to make sure it hasn't lost any data in the crash and possibly rewind some state changes in related modules.

> That's the theory yes, but I was asking about real life. Did those early microkernel systems actually deliver?

According to Wikipedia “[MINIX] can also withstand driver crashes. In many cases it can automatically restart drivers without affecting running processes. In this way, MINIX is self-healing and can be used in applications demanding high reliability”.

While this kernel was originally written to teach kernel design, all Intel chipsets post-2015 are running MINIX 3 internally as the software component of the Intel Management Engine.

Another widely deployed microkernel is L4, I assume this has similar capabilities, as it also puts most things in user space and is used for mission critical stuff.

> Also, for anything stateful, like a filesystem, simply relaunching it may not be sufficient.

True, but simply rebooting when the kernel process crashes due to buggy driver code won’t be sufficient either :)

FYI when Apple introduced extended attributes their AFP (network file system) did have a bug that made the kernel (and thus entire machine) crash for certain edge cases involving extended attributes.

In that case, had their AFP file system been a user space process, I may still have lost data, but it would have saved me from dozens of reboots.

Post reply on HN