Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Monorepos: Please don’t

medium.com

Post reply on HN
Print view View original

271–280 of 402 posts

Re: Monorepos: Please don’t

#271

Post by r_deckard »

gitgud wrote: post #240
forkerenok wrote: post #154

Earlier quoted context omitted.

And no, you cannot split a monorepo into a polyrepo easily. Been there, done that. _The reason is that working in a monorepo versus multiple repos influences the architecture quite a lot and the monorepo leads to very unclear boundaries. I think you are conflating a monorepo (where boundaries can still be established, e.g. via a module isolation mechanism specific to the stack used) with a "monoproject"/"monomodule",…

If the projects within the monorepo are decoupled and have clear boundaries then why not have them in separate repositories?... In my opinion monorepos make refactoring dependant projects much easier. However it is much harder to establish and enforce clear boundaries...

i have a monorepo that contains a few different early stage frontend web projects that does not interact with each other at all. They do however uses a shared component library that is also placed inside the monorepo. Tools like yarn workspaces makes sharing the library easy if the projects are located on the same repo.

When I change something on the library, i could easily also run tests across all the projects that depends on it with the latest changes of the library and make sure that my change is not breaking things all over the place, which is also pretty nice.

I am not sure yet if using a monorepo is actually the best way deal with this kind of projects, but for now it feels better than having them on seperate repos and then having to deal with the complexity of sharing the library across repos by publishing it somewhere or using git submodules or something.

Re: Monorepos: Please don’t

#272

Post by bborud »

ashelmire wrote: post #177
rossjudson wrote: post #148

Hilariously misguided. Pretty funny to read that the things I do every day are impossible. Monorepo and tight coupling are orthogonal issues. Limits on coupling come from the build system, not from the source repository. Yes, you should assume there is a sophisticated "VFS". What is this "checkout" you speak of? I have no time for that. I am too busy grepping the entire code base, which is apparently not possible. If…

“why on earth would google invest enormous effort constructing an entire ecosystem around a monorepo?” Didn’t google have a monorepo before git was created? And was created by academics? Legacy and momentum have a strong influence on the future. Hasn’t google also built a lot of tools for the monorepo and dedicates employees to it? That’s exactly the issue this article is about. From an external perspective, the spee…

If you think companies like Google are slow, you should look at the enterprise world. I do some technical due diligence work from time to time, and although things have gotten better of the years it wasn't that long ago that there was a more than 50% chance of seeing companies that only have source code repositories at a team level and 20% chance of them not even using a VCS to manage code.

I work for a company now where top management doesn't even understand what a repository is and what role it plays in software development.

Yes it is that bad in much of the "entreprise" world.

Yes, Google had a monorepo before git was in widespread use. They used Perforce while I was there, which was a miserable, miserable experience. It only worked because they poured engineering effort into making it somewhat tolerable.

I think it would be wrong to say Google chose a monorepo because it was the best choice. To be honest, I don't think they really planned how to deal with many thousands of developers when they made the choice. They just did what seemed to make sense at the time and then had to make it work as the challenges started to mount.

Re: Monorepos: Please don’t

#273

Post by jvolkman »

tensor wrote: post #62
nomel wrote: post #8

Earlier quoted context omitted.

The title should have been "Monorepos: Please don't, at scale", but I suspect it was made intentionally controversial.

Which large companies use polyrepos? Google, Facebook, Microsoft, Uber, all use monorepos.

Amazon

Re: Monorepos: Please don’t

#274

Post by bborud »

mrgriffin wrote: post #14

My problem with polyrepos is that often organizations end up splitting things too finely, and now I'm unable to make a single commit to introduce a feature because my changes have to live across several repositories. Which makes code review more annoying because you have to tab back and forth to see all the context. It's doubly frustrating when I'm (or my team is) the only people working on those repositories, becaus…

That's not really that different in a monorepo since you often need reviews from the same number of people anyway.

I once had to wait for 9 months to get a complex change through in a monorepo setting because of all the people involved, the number of stuff it touched and the fact that everything was constantly in flux so I spent half my time tracking changes. I'm not saying it would have been faster in a polyrepo. I'm saying that complex changes are complex regardless of how the source is organized.

I do however think that polyrepos forces you to be more disciplined and that it is easier to slip up in a polyrepo and turn a blind eye to tighter couplings.

Re: Monorepos: Please don’t

#275

Post by PeterisP »

tigershark wrote: post #182
avar wrote: post #170

Earlier quoted context omitted.

It'll be really typical for a gui/server to want to share some is_valid_payload() function. The client to validate it before sending, and for the server to do its own validation. If it's a monorepo your PR might be a 2 line patch to that function, then adding the GUI and server code. If you split it you'll first need to have a PR on the "validation-lib" repo, then once that gets in a PR on the "server" repo, bumping…

These are two separate functions why would you ever want a function that checks both gui and server? The gui validation logic belongs to the gui layer, the server validation logic to the server layer. If you have a function that contains logic from both layers there is something seriously wrong with your design.

The classic reason for any validation is that you want the validation to be done in the frontend (to save a network roundtrip and provide better, immediate feedback), on the backend (so that if the frontend is compromised and maliciously circumvents that validation, it still gets validated), and both of the validations to be the same to prevent inconsistencies.

A good way to fulfil those requirements is to have the exact same function available in both places.

Re: Monorepos: Please don’t

#276

Post by Lio »

adrianN wrote: post #116
0xFACEFEED wrote: post #98

Earlier quoted context omitted.

> You can't split monorepos after the fact, at least not without immense costs. Sure you can. The difficulty of doing so depends on many (many) factors. If your team does their job well then the costs won't be immense. It might be annoying, but not that hard. Speaking in absolutes or platitudes solves nothing. Sometimes monorepos make sense. Sometimes polyrepos make sense. It's entirely dependent on what your company…

Of course, if everybody is very diligent in keeping things in the monorepo distinct and independent, then it's easy to split it later on. But relying on constant diligence doesn't work out in the long run in my experience.

It's not like polyrepos solve the long term diligence problem though.

E.g. if relying solely on a package manager to to keep coupled things in lock step, you need make sure that version numbers are kept up to date for every little change made to every library.

You can easily end up with a situation where someone in another team makes a small change but doesn't change the lib version number. That's a people issue but it does happen.

You can get round that by using a repo SHA but now you two things to keep up to date for every library.

Like wise you'll have to be diligent in versioning APIs. Anecdotally I've found it easier to keep things in lock step when in a single repo and using a single pull request for each story than I have where separate teams have to keep separate repos in sync.

Both work but the monorepo approach worked better for the projects I've worked on. It just lead to less moving parts and more repeatability when there's a single SHA to watch.

I also have been luck enough that I haven't worked on a project so large that we couldn't build a monorepo on a single machine with "normal" build tooling.

Re: Monorepos: Please don’t

#277

Post by marcinkuzminski »

mrgriffin wrote: post #14

My problem with polyrepos is that often organizations end up splitting things too finely, and now I'm unable to make a single commit to introduce a feature because my changes have to live across several repositories. Which makes code review more annoying because you have to tab back and forth to see all the context. It's doubly frustrating when I'm (or my team is) the only people working on those repositories, becaus…

The multi-repository code review is an interesting concept. Here at RhodeCode we're actually working on such solution to implement. This is in first to solve our internal problem of release code-review spanning usually two projects at once.

This is a hard and complex problem. Especially how to make code-review not too messy if you target 5-8 repos at once.

Re: Monorepos: Please don’t

#278

Post by andy_ppp »

scarface74 wrote: post #226
andy_ppp wrote: post #145

Earlier quoted context omitted.

What are you talking about! In my perfect micro services world I just have these enforced bounded contexts that are so perfectly designed they never need to change. Consequently all parts of the system are perfectly independent snowflakes that can be deployed without thinking about any other parts of the system. It’s beautiful really when you think about the mess that things were before we could do this!

While I know you are being sarcastic, I really have heard bushy tailed young “architects” say something similar who just read about Domain Driven Design and then decided they were trying to “educate us”.

Oh I worked on a project like this, which still hasn’t launched any software yet 5 months after I left...

Re: Monorepos: Please don’t

#279

Post by notacoward »

yowlingcat wrote: post #269
drugme wrote: post #264

Earlier quoted context omitted.

I found it to be neither horseshit nor irresponsible. A bit overdrawn and skewed in some of its arguments, perhaps. But then again... so was your critique. For example: We had to migrate a polyrepo to a monorepo and it was not fun because it was a migration that should have never had to be done in the first place s/polyrepo/monorepo/ in the above and you have an assertion of about equal plausibility and weight.

No, it is horseshit. 99% of companies will never hit big company VCS scaling issues, and once they do, they're on their own. To characterize that scale as common is one of the most embarrassing failures of modern software engineering. People are so embarrassed to use well worn tooling and accept that large scale is both uncommon and something that doesn't invalidate tried and true patterns for smaller scales. It's ut…

> 99% of companies will never hit big company VCS scaling issues

A much higher percentage of developers will. Number of companies is not a good metric for whether a topic is worthy of discussion.

Re: Monorepos: Please don’t

#280

Post by jen20 »

kiallmacinnes wrote: post #267
thedufer wrote: post #241

Maybe its just that the author's cutoff is at the wrong team size, but the monorepo I work on (with ~150 devs) has almost none of the problems presented. Unreasonable for a single dev to have the entire repo? I'm looking at a repo with ~10 million LoC and ~1.4 million commits. I have 74 different branches checked out right now. Hard drives are cheap . Code refactors are impossible? I reviewed two of those this mornin…

In theory, yes - if the underlying repo changes, code review should start over. In practice though, it's a terrible idea ;) Part of code review is to ensure the code "fits" with all other merged code - so a re-review is "needed" when other changes merge. E.g. if I merge a refactor that changes everything from Pascal case to 100% SHOUTING, reviews now need to take this into account. In practice, this doesn't happen -…

> if I merge a refactor that changes everything from Pascal case to 100% SHOUTING, reviews now need to take this into account.

To be fair, if you get away with merging that refactor, the review that needed more attention was of that refactor ;-)

Post reply on HN