Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Monorepo is great if you're really good

yosefk.com

Post reply on HN
Print view View original

111–120 of 159 posts

Re: Monorepo is great if you're really good

#111

Post by kevan »

joshschreuder wrote: post #108
DanFeldman wrote: post #28

My team just switched to a monorepo. It's been only a few weeks, so I can't claim any results yet, but we've lived w/ the pain of poly-repo for long enough that we were ready to invest in a single repo. We've spent a lot of time building and iterating a unified ci/cd environment to support the new repo. Previously each project had it's own test/deploy/build/publish story and usually it's own jenkins project. Now, eac…

How do you handle building changes to just one of those projects? Can Jenkins do that (easily)? I think that's the big thing that always puts me off monorepo... We'd basically be going from ten 5 minute builds to one 50 minute build if it wasn't possible to do incremental builds. IIRC Google and MS have purpose built tools that do impact detection to work out what to build for their monorepos to keep build times down…

If you're doing a monorepo I think it's strongly implied that you'll also use a build system (Blaze/Brazil/BuildXL etc) that has granular compilation units and output caching so build time doesn't scale linearly with the company's total codebase.

It's definitely important to consider before jumping in. Going from 5m to 50m compile times would be a major issue for me.

Re: Monorepo is great if you're really good

#112

Post by pklausler »

cbanek wrote: post #3

> the entire code base got forked, and the entire org is now held hostage by the dumbass. > Of course in a Good team, needless dependencies would be weeded out in code reviews, and a Culture would evolve over time avoiding needless dependencies. Really, the one consistent thing is that if you have a good team, you'll make it work no matter what tech or decisions you make (assuming you're also good enough to know when…

Observation: the quality of the work done by a team is too often a MIN or PRODUCT over the members of the team, not a SUM.

Re: Monorepo is great if you're really good

#113

Post by jjgreen »

gotofritz wrote: post #110
gen220 wrote: post #53

Earlier quoted context omitted.

We (~150 eng) build microservices in a polyglot environment (mostly Python, JS, and Go), all in a monorepo! We also build + deploy in containers with Jenkins, etc. The structure looks something like this: |-- third_party (third party libs) `-- python |-- libs (internally-written, common dependencies) `-- application_name |-- client (react app, connects to flask) |-- server (flask app, connects to services) `-- servic…

Organising apps by language seems weird...

I've seen this and it works rather well in that one can have per-language bottom level build setup (with autoconf for C, setuptools for Python, ...).

Re: Monorepo is great if you're really good

#114

Post by [deleted] »

jandrese wrote: post #83
jayd16 wrote: post #77

Earlier quoted context omitted.

Tell that to my (lack of) graphics drivers. You can say its political but as it stands its no where near apples to apples in terms of what Windows supports vs what Linux supports.

Which video card do you have that lacks drivers for Linux? Or do you need fully open source drivers that fully support 3D acceleration and computation?

[deleted]

Re: Monorepo is great if you're really good

#115

Post by gen220 »

jjgreen wrote: post #113
gotofritz wrote: post #110

Earlier quoted context omitted.

Organising apps by language seems weird...

I've seen this and it works rather well in that one can have per-language bottom level build setup (with autoconf for C, setuptools for Python, ...).

I agree with the GP that it seems weird, and with you that it has its benefits. Of course, other approaches have their own benefits.

Personally, I feel like the top-level directory ordering for a monorepo is somewhat arbitrary, in that you can argue for anything, but it probably doesn’t matter; especially if you have a decent build tool.

Re: Monorepo is great if you're really good

#116

Post by [deleted] »

nostrademons wrote: post #85
liability wrote: post #61

Earlier quoted context omitted.

So similar in principle to FUSE, but applied more broadly? Seems like a neat idea.

In practice it was less useful than people assumed, because: 1. Things like drivers and filesystems are usually written by a small handful of vendors, who already have rigorous engineering cultures (hardware is a lot less forgiving than say web design), and a large base of demanding users who will rapidly complain and/or sue you if you get it wrong. When was the last time you personally had a crash due to a driver or…

[deleted]

Re: Monorepo is great if you're really good

#117

Post by [deleted] »

nostrademons wrote: post #85
liability wrote: post #61

Earlier quoted context omitted.

So similar in principle to FUSE, but applied more broadly? Seems like a neat idea.

In practice it was less useful than people assumed, because: 1. Things like drivers and filesystems are usually written by a small handful of vendors, who already have rigorous engineering cultures (hardware is a lot less forgiving than say web design), and a large base of demanding users who will rapidly complain and/or sue you if you get it wrong. When was the last time you personally had a crash due to a driver or…

[deleted]

Re: Monorepo is great if you're really good

#118

Post by sagarm »

kevan wrote: post #111
joshschreuder wrote: post #108

Earlier quoted context omitted.

How do you handle building changes to just one of those projects? Can Jenkins do that (easily)? I think that's the big thing that always puts me off monorepo... We'd basically be going from ten 5 minute builds to one 50 minute build if it wasn't possible to do incremental builds. IIRC Google and MS have purpose built tools that do impact detection to work out what to build for their monorepos to keep build times down…

If you're doing a monorepo I think it's strongly implied that you'll also use a build system (Blaze/Brazil/BuildXL etc) that has granular compilation units and output caching so build time doesn't scale linearly with the company's total codebase. It's definitely important to consider before jumping in. Going from 5m to 50m compile times would be a major issue for me.

Err what is even the alternative? Even a makefile would provide incremental builds.

Re: Monorepo is great if you're really good

#119

Post by andrewprock »

detaro wrote: post #91
andrewprock wrote: post #89

Earlier quoted context omitted.

Unfortunately, breaking a 10 gig repo into 5 repos often means you have to now download five 2 gig repos.

That can still be better, e.g. when part of the problem are tools that work on a per-repo level and fall over/get slow if the repo gets to large.

How do these tools work when it comes to working on multiple repos? I imagine they either don't, or that it's worse than working on a single monorepo of the same size.

Re: Monorepo is great if you're really good

#120

Post by joshschreuder »

kevan wrote: post #111
joshschreuder wrote: post #108

Earlier quoted context omitted.

How do you handle building changes to just one of those projects? Can Jenkins do that (easily)? I think that's the big thing that always puts me off monorepo... We'd basically be going from ten 5 minute builds to one 50 minute build if it wasn't possible to do incremental builds. IIRC Google and MS have purpose built tools that do impact detection to work out what to build for their monorepos to keep build times down…

If you're doing a monorepo I think it's strongly implied that you'll also use a build system (Blaze/Brazil/BuildXL etc) that has granular compilation units and output caching so build time doesn't scale linearly with the company's total codebase. It's definitely important to consider before jumping in. Going from 5m to 50m compile times would be a major issue for me.

Good to know, thanks. Something I'll have to do a bit more reading up on.
Post reply on HN