Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Monorepos: Please don’t

medium.com

Post reply on HN
Print view View original

211–220 of 402 posts

Re: Monorepos: Please don’t

#211

Post by skj »

Aeolos wrote: post #194
tigershark wrote: post #179

Earlier quoted context omitted.

Because obviously the changes that you make in the gui are completely isolated from the changes you make on the server. When you are working on the gui the server code is just noise and vice versa. And it gets even worse when you use two different languages for the gui and the server.

> Because obviously the changes that you make in the gui are completely isolated from the changes you make on the server. In my experience, that is almost never the case. Often, the frontend requires a new endpoint or a modification to an existing endpoint. If you don't coordinate this change, you end up with a non-functional PR that cannot even be tested. Same happens when the backend proposes an endpoint change tha…

You make the endpoint first, and test it without the UI. What challenges do you foresee here?

Re: Monorepos: Please don’t

#212

Post by majewsky »

atq2119 wrote: post #200
tigershark wrote: post #179

Earlier quoted context omitted.

Because obviously the changes that you make in the gui are completely isolated from the changes you make on the server. When you are working on the gui the server code is just noise and vice versa. And it gets even worse when you use two different languages for the gui and the server.

This is true for systems where there is a well-defined protocol between GUI and servers and a proper versioning process in place, i.e. most "old-school" client/server systems. I expect lots of people on HN are working on systems with very tight coupling between client/GUI and server and no proper versioning between them, as is common in web applications. Hence the replies to the contrary: you're probably from quite d…

I think what matters in the end is Conway's law. Conway's is frequently misinterpreted as an observation when it's actually advice: Structure your applications/repos like you structure your teams. You're going to end up with that code structure anyway, so might as well save some time.

Re: Monorepos: Please don’t

#213

Post by bpicolo »

forty wrote: post #113
curtis wrote: post #32

My advice is that if components need to release together, then they ought to be in the same repo. I'd probably go further and say that if you just think components might need to release together then they should go in the same repo, because you can in fact pretty easily manage projects with different release schedules from the same repo if you really need to. On the other hand if you've got a whole bunch of component…

My rule of thumb is: if you need to do PRs in several repositories to do one features, you should probably merge the repositories. At work, we have code spread among a bunch of repositories, and having to link to the 2/3 related PRs in other repos is a major PITA, and even more so for the reviewers.

At a past job, I had to edit roughly 5 different repositories in order to do some trivial programming task (send an email or some such). It was quite easily the least productive / most demoralizing workflow I've ever experienced.

Context switching really sucks. You should aim to reasonably avoid it

Re: Monorepos: Please don’t

#214

Post by ryanbrunner »

tigershark wrote: post #185
avar wrote: post #183

Earlier quoted context omitted.

Imagine something like "curl" where a client needs to validate a manually provided request before making it. In any case, if you're nitpicking that example you're missing the point. The same would go for any number of other shared code you could imagine between a client/server trying that logically make up one program talking over a network.

I still can’t see how you would have a shared library for a C# gui and a Java server for example. Your communication layer would obviously live in both repositories. Even in case you are using the same language and you do have shared libraries then what is the problem? The shared libraries would surely be shared with other projects so it makes sense to have them in a separate repository.

In cases where there's a high degree of churn (i.e. early-stage startups) in shared libraries, updating those libraries can cause a large amount of busywork and ceremony.

If you had a `foo()` function shared between the GUI and the server (or two services on your backend, or whatever), in a monorepo your workflow is:

   - Update foo()
   - Merge to master
   - Deploy
In a polyrepo where foo() is defined in a versioned, shared library your workflow is now:

   - Update foo()
   - Merge to shared library master
   - Publish shared library
   - Rev the version of shared library on the client
   - Merge to master
   - Deploy client
   - Rev the version of shared library on the server
   - Merge to master
   - Deploy server
This problem gets even more compounded when your dependencies start to get more than one level deep.

I recently dealt with an incredibly minor bug (1 quick code change), that still required 14 separate PRs to get fully out to production in order to cover all of our dependencies. That's a lot of busywork to contend with.

Re: Monorepos: Please don’t

#215

Post by Vinnl »

tigershark wrote: post #195
Vinnl wrote: post #192

Earlier quoted context omitted.

Hmm, that's not really obvious to me. Sometimes the server has to deliver new data that is to be used in the GUI, so it's nice to be able to present those together in the same PR. If it then happens that the server-side changes do not match what you need in the GUI, it's relatively painless to add those changes in the same branch that hasn't yet been merged. In other words: although you can make changes in one withou…

You should always have a communication layer between the gui and the server. For example using protobuf you would update the proto definition (that can be in a shared repo) and when building the gui and the server the protobuf layer is regenerated. So the only place where you make your changes for the new data contract is the shared repo and the gui and server would automatically have the new changes.

So now we're at three repo's, one of which is shared by the other two, and changes will have to be coordinated over them. I fail to see how that is an improvement over having both in the same repo.

In the end, I think the other comments are right that it mostly depends on who's working on something. If it's different teams, then different repo's probably make sense. But if I'm responsible for both the back-end and the front-end, they're usually not isolated at all at least in terms of project requirements, and hence keeping them together makes sense.

(But of course, even then there's nuances. I think the article is mostly arguing against monorepo's as in company-wide monorepo's. I'm willing to believe Googlers that it works well for Google, and I'm not in a position to claim what it'd be like for other companies. Team-wide monorepo's for different parts of the same project, however, make a lot of sense to me.)

Re: Monorepos: Please don’t

#216

Post by rauhl »

inertiatic wrote: post #175
mrgriffin wrote: post #14

My problem with polyrepos is that often organizations end up splitting things too finely, and now I'm unable to make a single commit to introduce a feature because my changes have to live across several repositories. Which makes code review more annoying because you have to tab back and forth to see all the context. It's doubly frustrating when I'm (or my team is) the only people working on those repositories, becaus…

My current team managed to break a single "component" out into a separate repository. Then that repository broke into two, then those broke into other repositories, until we've eventually have around 10 or so different repositories that we work on every day. An average change touches 4 of them, and touching one of them triggers on average releases on 2 or 3 of them. Even building these locally is super tedious, becau…

I’ve come to the conclusion that an organisation should ideally have no more than one primary repo, with maybe a handful of ancillary repos for stuff that really doesn’t make sense in the primary. What does ‘organisation’ mean there? Well, it could mean a company, or a team, or a division. Just as software conforms to organisational structure (Conway’s Law), so too should repo structure.

Once you start having lots of peer repos being worked on within the same organisation on a daily basis, you know that you’ve partitioned far too far, and you need to roll back.

Otherwise one ends up in exactly the position you’re in. The ultimate slippery-slope end-state would be hilariously bad: a repo for each ASCII character, with repos for each word or symbol constructed out of those characters, with repos for each function constructed out of those words & symbols, with repos for each module constructed out of those functions, with repos for each system constructed out of those modules, with any change requiring a massive, intricate, failure-prone dance in order to update anything, all while patting oneself on the back about how one has avoided complexity.

Noöne sane would argue for that situation, and yet I’ve seen smart people argue that requiring coördinated changes to half a dozen repos is fine & dandy.

Re: Monorepos: Please don’t

#217

Post by dsumenkovic »

gusmd wrote: post #84
Mxtetris wrote: post #45

Earlier quoted context omitted.

Can a monorepo support module- or subdirectory-level ownership controls? Or do teams using a monorepo just do without them? Partially answering my own question: SVN, recommended in a prior comment [0], supports path-based authorization [1]. But what about teams using another version control system? [0] https://news.ycombinator.com/item?id=18810313 [1] http://svnbook.red-bean.com/en/1.5/svn-book.html#svn.serverc...

If I remember correctly, Gitlab has introduced some sort of ownership control where you can say who owns what directories for things like approving merge requests that affect those directories.

Hey, did you mean of assigning approvers based on code owners [1]? You can find more info about Code Owners and syntax at the documentation [2].

[1] https://gitlab.com/gitlab-org/gitlab-ee/issues/1012 [2] https://docs.gitlab.com/ee/user/project/code_owners.html

Re: Monorepos: Please don’t

#218

Post by fnord123 »

jhasse wrote: post #130
jogjayr wrote: post #107

Earlier quoted context omitted.

> Monorepos allow you to do very bad hacks (I need this other component over there; let me just put in a Symlink. Done.). Why would you put in a symlink? You could just provide a path to the actual component and import it into your project. > the worst thing you can get that you'll have to assemble multiple distinct, well-encapsulated (in terms of project structure) things into one When you have multiple repos, you a…

> Why would you put in a symlink? You could just provide a path to the actual component and import it into your project. Where do I need to put the path again? Ah what the heck, I'll just add a symlink inside a folder that's already somewhere in the build definitions.

What language and build tool is this that you're using?

I don't know anyone who has abused Maven or Cargo or Go like this. And I don't imagine Visual Studio Solutions for C# are used like this.

Is there an underlying disagreement based on JS/Ruby/Python scriptish coding (which creaks when a lot of developers work on it) vs C and C++ (which have astonishingly bad build system stories) vs big-iron languages that don't sweat when in a monorepo.

Re: Monorepos: Please don’t

#219

Post by [deleted] »

harunurhan wrote: post #119

I have worked with polyrepo madness... I do remember doing commits to up to 5 different repos just for a feature. And to roll this feature to prod, few of these repos had to go through release process. On top of everything we couldn't really write tests to ensure if the feature works. The best we could do is write tests on the "user facing" repo and keep fixing and releasing others until those pass. Well, I am sure m…

[deleted]

Re: Monorepos: Please don’t

#220

Post by rauhl »

adrianN wrote: post #90
0xFACEFEED wrote: post #52

At least the author gave us the courtesy of italicizing his broken assumption from the outset of the post. > Because, at scale, a monorepo must solve every problem that a polyrepo must solve, with the downside of encouraging tight coupling, and the additional herculean effort of tackling VCS scalability. Right. But you have to get to "scale" first (as it relates to VCSs). Most companies don't. Even if they're success…

You can't split monorepos after the fact, at least not without immense costs. You can always just put all your small repos into a big one.

> You can't split monorepos after the fact, at least not without immense costs.

That has not been my experience at all. At a previous employer, we did exactly that with a multi-language library. In fairness, having multiple languages enforced fairly good directory structure in our single repo. But isn’t that the real point: good structure makes life easier, period. The thing is, going into a project you often don’t know what the right structures are yet. Creating a new repo for each component you think you need ossifies those choices, making it far more difficult to walk back on them later on (first because you may not even see the architectural mistake, second because the maintainers of that component will have an investment in its existence).

> You can always just put all your small repos into a big one.

In my experience, that’s harder, precisely because over time so much tooling has been built into each repo to manage builds, images, deployment &c.

I’ve worked in monorepos & I’ve worked in multrepos, and so far my experience has been that monorepos enable faster velocity and more-maintainable software. I’ve not (yet) worked at Google- or Facebook-scale, though, and I’m completely open to the idea that at that scale a team really does need lots and lots of repos, and tooling to stitch them all together.

Post reply on HN