Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Monorepos: Please don’t

medium.com

Post reply on HN
Print view View original

201–210 of 402 posts

Re: Monorepos: Please don’t

#201

Post by notacoward »

joshuamorton wrote: post #74
wtracy wrote: post #60

I'm not familiar with how monorepos work in practice, but it seems obvious to me that it's going to complicate everyday tasks. Ready to commit? Whoops, another team made a bunch of commits to their project, and you need to rebase your project before you can commit. (I'm having flashbacks to Clearcase already.) Need to roll back the last two commits you made? Sure, that takes two seconds--oh, wait, another team made m…

Why would you need to rebase or cherrypick unless you and the other team were touching the same files?

Problem is, places that use monorepos also tend to have whole teams full of people who feel entitled to f#ck with stuff across the entire repo, often without going through the normal review or other processes for each component. Thus it's not uncommon to commit code early in the day so that you can build packages for system test, find a problem in system test, then come back later the same day to find one of those randos has already "fixed" your code to adhere to some new standard they decided on at lunchtime. Now reverting only your own commit is a mess, and reverting theirs as well invites a s###storm of epic proportions because they're higher-caste than you.

Re: Monorepos: Please don’t

#202

Post by dtech »

rossjudson wrote: post #148

Hilariously misguided. Pretty funny to read that the things I do every day are impossible. Monorepo and tight coupling are orthogonal issues. Limits on coupling come from the build system, not from the source repository. Yes, you should assume there is a sophisticated "VFS". What is this "checkout" you speak of? I have no time for that. I am too busy grepping the entire code base, which is apparently not possible. If…

3) A monorepo with significant investment in ecosystem and tooling is a better choise than a polyrepo

For other (smaller) companies, polyrepo might be the better choice because [significant investment in ecosystem and tooling] is not appealing, and the investments of Google et al. have not leaked through sufficiently into general available tools. Some headway is being made in the latter [1], so monorepo might be the "obvious" best choice in 10 years or so.

[1] For example, Git large file support is mostly from corporate contributors https://git-lfs.github.com/ https://github.com/Microsoft/VFSForGit

Re: Monorepos: Please don’t

#203

Post by cmrdporcupine »

It's worth pointing out that while Google has a monorepo in Google3, it also doesn't at the same time. We have are other projects such as Android, ones based on Chrome, etc. that are composed of multiple git projects and use repo to manage and sync.

Re: Monorepos: Please don’t

#204

Post by svat »

wtracy wrote: post #60

I'm not familiar with how monorepos work in practice, but it seems obvious to me that it's going to complicate everyday tasks. Ready to commit? Whoops, another team made a bunch of commits to their project, and you need to rebase your project before you can commit. (I'm having flashbacks to Clearcase already.) Need to roll back the last two commits you made? Sure, that takes two seconds--oh, wait, another team made m…

See Hyrum Wright's talks on Large-Scale Changes at Google: https://www.youtube.com/watch?v=ZpvvmvITOrk (CppCon2014), https://www.youtube.com/watch?v=TrC6ROeV4GI (CppCon 2018), and paper by Googlers on ClangMR (http://www.hyrumwright.org/papers/icsm2013.pdf). It's great!

And if you'd like to learn more about how monorepos work in practice there's a couple of papers: https://cacm.acm.org/magazines/2016/7/204032-why-google-stor... and https://ai.google/research/pubs/pub47040

(Also worth reading: http://danluu.com/monorepo/)

Re: Monorepos: Please don’t

#205

Post by jenscow »

tigershark wrote: post #182
avar wrote: post #170

Earlier quoted context omitted.

It'll be really typical for a gui/server to want to share some is_valid_payload() function. The client to validate it before sending, and for the server to do its own validation. If it's a monorepo your PR might be a 2 line patch to that function, then adding the GUI and server code. If you split it you'll first need to have a PR on the "validation-lib" repo, then once that gets in a PR on the "server" repo, bumping…

These are two separate functions why would you ever want a function that checks both gui and server? The gui validation logic belongs to the gui layer, the server validation logic to the server layer. If you have a function that contains logic from both layers there is something seriously wrong with your design.

If you have the same functionality that can't be re-used (for no reason), then I'd call that a design flaw.

I'll need a few more validation functions for each clients. I don't want to write+maintain multiple functions that do the same thing, even if it's just copy+paste.

It's "data" validation. So let's put that in the "data layer" repo.

We now have, at least:

- Server

- Web (GUI)

- Android

- iOS

- Data

- More clients?

We'll also have branches for each development task. How do we know what branch the other branches should use? One "simple" feature can easily spread over multiple repos. Does each repo refer to the repo+branch it depends upon (don't forget to update the references when we merge!), or we add a "build" repo which acts as the orchestrator?

Most PRs will need to be daisy chained - who reviews each one? Will they get comitted at the same time?

How do we make the builds reproducible? commit hashes? tags? ok, we now need to tag each repo, and update the references to point to that tag/hash... but that changes the build.

Well, I'm glad our code base is split over multiple repos because "scalability".

Re: Monorepos: Please don’t

#206

Post by eadmund »

avar wrote: post #170
tigershark wrote: post #141

Earlier quoted context omitted.

Not always. It makes absolutely sense to have a repository for the gui and one for the server. When writing a new feature you usually write some gui code and some server code and create different pull requests. I think monorepos are seriously wrong and I completely agree with this article.

It'll be really typical for a gui/server to want to share some is_valid_payload() function. The client to validate it before sending, and for the server to do its own validation. If it's a monorepo your PR might be a 2 line patch to that function, then adding the GUI and server code. If you split it you'll first need to have a PR on the "validation-lib" repo, then once that gets in a PR on the "server" repo, bumping…

> If it's a monorepo your PR might be a 2 line patch to that function, then adding the GUI and server code.

> If you split it you'll first need to have a PR on the "validation-lib" repo, then once that gets in a PR on the "server" repo, bumping the "validation-lib" version dependency, and finally a PR on the "gui" repo bumping the dependency for both "validation-lib" and "server" (for testing etc.). That's before you need do deal with the circular dependency that "server" also wants "gui" for its own "I changed my server code, does the GUI work?" testing.

The above is exactly why I am so firmly opposed to multirepo[0]-first. And it's really just a throwaway example: a real change would involve multiple different library and executable repos, all having separate PRs. And then there's the relatively high risk of getting a circular incompatibility.

This can be worth the cost, for organisational reasons. But until you need it, don't do it. It's very easy to split a git repo into multiple repos, each retaining its history (using git filter-branch). Don't incur the pain until you need to, because honestly, you're not likely to need to. You're probably not going to grow to the size of Google. Heck, most of Google runs in one monorepo, with a few other repos on the side: if they can make it work at their scale, so can you. And if, as the odds are, you never grow to their size, then you'll never have wasted time engineering a successful multirepo system instead of delivering features to your business & customers.

0: 'polyrepo,' really? https://trends.google.com/trends/explore?date=all&q=multirep... clearly shows that 'multirepo' is term.

Re: Monorepos: Please don’t

#207

Post by NicoJuicy »

Since i knew that Google did it, i have started to think about it, a lot.

And mono-repos really do make sense ( a lot) when you need them tied together. Finding errors in your console immediatly without version numbers gets the job faster done.

There are other ways though, like if you use dot net. A mono repo that creates nuget packages and projects that pull the latest build of them into their solution. This way, external parties can re-use the same components.

On a beta version, that releases new nuget components, if there is a file change ( and so a version update), notify the external parties.

Have one website which mentions the schedule of an update on the live version to reduce email traffic. Oldskool, but it seems to work.

Re: Monorepos: Please don’t

#208

Post by eadmund »

dtech wrote: post #202
rossjudson wrote: post #148

Hilariously misguided. Pretty funny to read that the things I do every day are impossible. Monorepo and tight coupling are orthogonal issues. Limits on coupling come from the build system, not from the source repository. Yes, you should assume there is a sophisticated "VFS". What is this "checkout" you speak of? I have no time for that. I am too busy grepping the entire code base, which is apparently not possible. If…

3) A monorepo with significant investment in ecosystem and tooling is a better choise than a polyrepo For other (smaller) companies, polyrepo might be the better choice because [significant investment in ecosystem and tooling] is not appealing, and the investments of Google et al. have not leaked through sufficiently into general available tools. Some headway is being made in the latter [1], so monorepo might be the…

> For other (smaller) companies, polyrepo might be the better choice because [significant investment in ecosystem and tooling] is not appealing

That's not the choice, though: significant investment in tooling is a function of codebase size. In my own experience, polyrepos require more tooling, because you're not just dealing with files & directories, you're also dealing with repos (& probably PRs & issues & other stuff in a forge).

Re: Monorepos: Please don’t

#209

Post by ashelmire »

titanomachy wrote: post #187
ashelmire wrote: post #177

Earlier quoted context omitted.

“why on earth would google invest enormous effort constructing an entire ecosystem around a monorepo?” Didn’t google have a monorepo before git was created? And was created by academics? Legacy and momentum have a strong influence on the future. Hasn’t google also built a lot of tools for the monorepo and dedicates employees to it? That’s exactly the issue this article is about. From an external perspective, the spee…

Does Google require more engineers to support their build system than they would with a polyrepo? That question is not trivial to answer, IMO.

Any is more than 0 though. In my experience (probably shared by many devs), polyrepos don’t require a team, or even a single person, dedicated to version control. It’s a minor part of the software management (usually: “mind if I create a new repo for this?” “Yes/no”).

It does affect dependency management but no more than any external dependency.

Re: Monorepos: Please don’t

#210

Post by majewsky »

forty wrote: post #113
curtis wrote: post #32

My advice is that if components need to release together, then they ought to be in the same repo. I'd probably go further and say that if you just think components might need to release together then they should go in the same repo, because you can in fact pretty easily manage projects with different release schedules from the same repo if you really need to. On the other hand if you've got a whole bunch of component…

My rule of thumb is: if you need to do PRs in several repositories to do one features, you should probably merge the repositories. At work, we have code spread among a bunch of repositories, and having to link to the 2/3 related PRs in other repos is a major PITA, and even more so for the reviewers.

Just because things change in tandem, that does not mean that they're all the same thing. When I add a new function to my backend service, all frontends that consume its API also need to be adjusted. But that doesn't mean that the backend service, its command-line clients and its web GUI client should live in the same repo.
Post reply on HN