Monorepos: Please don’t • Hacker News Board

Re: Monorepos: Please don’t

#151

Post by lclarkmichalek » Thu, Jan 03, 2019, 10:04 AM UTC

wtracy wrote: post #60

I'm not familiar with how monorepos work in practice, but it seems obvious to me that it's going to complicate everyday tasks. Ready to commit? Whoops, another team made a bunch of commits to their project, and you need to rebase your project before you can commit. (I'm having flashbacks to Clearcase already.) Need to roll back the last two commits you made? Sure, that takes two seconds--oh, wait, another team made m…

> Ready to commit? Whoops, another team made a bunch of commits to their project, and you need to rebase your project before you can commit. (I'm having flashbacks to Clearcase already.)

If they merge cleanly, it's not an issue. If they don't, you need to fix the merge conflict. The work you need to do is proportional to the number of merge conflicts, which isn't special to monorepos.

> Need to roll back the last two commits you made? Sure, that takes two seconds--oh, wait, another team made multiple commits that got interleaved with yours. Have fun cherry picking the files you want to revert.

Again, only an issue if the changes are on the same files. It can be a bit of a pain to revert a stack of diffs, but if it's just a random commit with no other relevent commits to the file, very easy.

Re: Monorepos: Please don’t

#152

Post by tome » Thu, Jan 03, 2019, 10:06 AM UTC

Xylakant wrote: post #125

ajuc wrote: post #31

I call bullshit on "our repository is too big for one machine". Seriously, you have over 1 TB of code and 100 people wrote it?

adding raw versions of binary assets (designs, video, ...) can quickly lift a repo beyond a TB. Now, you could say "don't do that", but there's valid use cases where you'd want to track all binary assets as part of the development cycle.

Ouch, well, yes that is a very good situation in which not to take the "mono" part of "monorepo" too seriously.

Re: Monorepos: Please don’t

#153

Post by m0zg » Thu, Jan 03, 2019, 10:07 AM UTC

"Scalability" issues aren't encountered until your repo has many millions of LOCs and a lot of churn. For 99.99% of organizations this is not an issue and will never be an issue.

Re: Monorepos: Please don’t

#154

Post by forkerenok » Thu, Jan 03, 2019, 10:08 AM UTC

bad_user wrote: post #109

curtis wrote: post #32

My advice is that if components need to release together, then they ought to be in the same repo. I'd probably go further and say that if you just think components might need to release together then they should go in the same repo, because you can in fact pretty easily manage projects with different release schedules from the same repo if you really need to. On the other hand if you've got a whole bunch of component…

Components might need to be released “together”, but if they are worked on by different teams, it means they’ll have a different release process, as in different timeline, different priorities. First of all this is normal, because otherwise the development doesn’t scale. In such a case the monorepo starts to suck. And that’s the problem with your philosophy ... it matters less how the components connect, it matters m…

And no, you cannot split a monorepo into a polyrepo easily. Been there, done that. _The reason is that working in a monorepo versus multiple repos influences the architecture quite a lot and the monorepo leads to very unclear boundaries.

I think you are conflating a monorepo (where boundaries can still be established, e.g. via a module isolation mechanism specific to the stack used) with a "monoproject"/"monomodule", where is no modularization at all.

Edit: expanded wording

Re: Monorepos: Please don’t

#155

Post by twic » Thu, Jan 03, 2019, 10:11 AM UTC

harunurhan wrote: post #119

I have worked with polyrepo madness... I do remember doing commits to up to 5 different repos just for a feature. And to roll this feature to prod, few of these repos had to go through release process. On top of everything we couldn't really write tests to ensure if the feature works. The best we could do is write tests on the "user facing" repo and keep fixing and releasing others until those pass. Well, I am sure m…

Monorepo = all applications in one repo

Polyrepo = each application in its own repo

Whatever-madness-you-had-repo = each application across multiple repos

I'm sorry to hear you had to suffer what you did, but that was not the only alternative to a monorepo!

Re: Monorepos: Please don’t

#156

Post by username90 » Thu, Jan 03, 2019, 10:14 AM UTC

With the right tooling for both types each directory in a monorepo is equivalent to a repository in a multirepo setup. The only difference is that in the monorepo it is easier to create new repos and dependencies between repos (just add a directory in a commit or add a dependency on another directory).

The author of this piece apparently think that the ease to work in a monorepo is a bad thing, I disagree. I think that being able to treat repositories as easily as directories is awesome since it is a lot simpler so requires a lot less training for your devs to understand.

Re: Monorepos: Please don’t

#157

Post by tome » Thu, Jan 03, 2019, 10:14 AM UTC

hermitdev wrote: post #48

sftwds wrote: post #46

>Scaling a single VCS to hundreds of developers, hundreds of millions lines of code... Maybe I am way out of my element here, but is this a common problem? Do companies with only “hundreds of engineers” really have “hundreds of millions of lines of code”?

From personal experience, it can happen. At one point, I was personally responsible for about 2 million lines of code. Over several years, I was able to reduce it to about 500k through generous use of code generation for ORM type work. The generated code never ended up in VCS, but the generator and model did. Certainly helped checkout/update times as there was several thousand fewer files to deal with. I was one of a…

> I was one of about 900 engineers at a financial company of about 1500 employees at the time.

Can you say which company it is, or give a few more details? I'm fascinated to know which financial company can consist of 60% engineers!

Re: Monorepos: Please don’t

#158

Post by pcj-github » Thu, Jan 03, 2019, 10:18 AM UTC

vikingcaffiene wrote: post #61

I am sure this author means well but I respectfully disagree with this advice. The author is arguing against the monorepo approach and then proceeds to list out some of the most successful software companies on earth as reasons NOT to do it. The reason they were able to get to their lofty heights was in some part because they used a monorepo. The biggest advantage of a monorepo is you can move quickly and understand…

Yup... This. You saved me 20m from writing this same/similar reply.

Tragically this article will be subsequently be cited by countless software managers that retain a fear of monorepos for some of the reasons cited here; junior (& sr devs) will go along with it out of not wanting to stick their necks out, and the cycle of pain will continue.

Re: Monorepos: Please don’t

#159

Post by IshKebab » Thu, Jan 03, 2019, 10:19 AM UTC

malkia wrote: post #69

I do really like mono-repos, but google's other significant new project: fuchsia - is set-up as multi-git repo (and I believe chromium too, maybe android (haven't checked)). For fuchsia, they use a tool called "jiri"[1] to update the repos, previously (and maybe still in use) is the "gclient" sync tool [2] way from depot_tools[3] [1] - https://fuchsia.googlesource.com/jiri/ [2] - https://chromium.googlesource.com/chr…

Funny that there are so many reimplementations of git submodules but with support for "just give me HEAD" - Google has two (jiri and repo), my company has a home-grown one too.

Re: Monorepos: Please don’t

#160

Post by lrem » Thu, Jan 03, 2019, 10:19 AM UTC

Mxtetris wrote: post #45

WorldMaker wrote: post #16

Earlier quoted context omitted.

It's an interesting social problem in how you manage those project / library / repository boundaries. On the flipside, though, it's been well documented that among many of the major monorepos those boundaries still exist, they just become far more opaque because no one has to track them. You find the weird gatekeepers in the dark that spring out only when you get late in your code review process because you touched "…

Can a monorepo support module- or subdirectory-level ownership controls? Or do teams using a monorepo just do without them? Partially answering my own question: SVN, recommended in a prior comment [0], supports path-based authorization [1]. But what about teams using another version control system? [0] https://news.ycombinator.com/item?id=18810313 [1] http://svnbook.red-bean.com/en/1.5/svn-book.html#svn.serverc...

In Google, this is pretty explicit with a plaintext OWNERS file in the directory. Internal IDEs not only have an understanding of that, but can automatically suggest a minimal set of reviewers in a possibly close time zone and not out of office.