Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Monorepos: Please don’t

medium.com

Post reply on HN
Print view View original

91–100 of 402 posts

Re: Monorepos: Please don’t

#91

Post by outsomnia »

0xFACEFEED wrote: post #52

At least the author gave us the courtesy of italicizing his broken assumption from the outset of the post. > Because, at scale, a monorepo must solve every problem that a polyrepo must solve, with the downside of encouraging tight coupling, and the additional herculean effort of tackling VCS scalability. Right. But you have to get to "scale" first (as it relates to VCSs). Most companies don't. Even if they're success…

The author talks about proponents of monorepos, but I thought when I read it: actually they are victims of monorepos trying to explain to themselves as much as anyone why they choose to suffer with them. (Actual reason: for $$$).

Nobody would choose to drag around every historical afterthought in the development sequence of long forgotten software going back three decades that no longer builds with current tools, just so they can work on a small library off in a corner. Software is getting written and added to these monorepos at a much faster rate than hardware and networks are able to hide the bloat-upon-bloat growth of them.

Re: Monorepos: Please don’t

#92

Post by chvid »

Does a lot of the pain from a monorepo come from trying to use a tool - Git - that is explicitly designed to support distributed repositories? Wouldn't things be easier if you used eg. Subversion instead? That is a tool that was designed around a client/server paradigm and had a single repository as its main use case.

Re: Monorepos: Please don’t

#93

Post by btschaegg »

clhodapp wrote: post #56
naniwaduni wrote: post #34

Earlier quoted context omitted.

> As a final observation, you can split big repositories into smaller ones quite easily (in Git anyway) but sticking small repositories together into a bigger one is a lot harder. So start out with a monorepo and only split smaller repositories out when it's clear that it really makes sense. If you only need to do this once, subtree will do the job, even retaining all your history if you want. I'm not sure what the e…

To split, you can duplicate the repo and pull trees out of each dupe in normal commits.

In principle: Yes.

In practice, I can tell you from first-hand experience that this isn't all that simple in bigger, organically grown cases (you'll have many other things to consider if you want to keep the history in a useful way). Especially the broken branching model of SVN and co. is a problem here: In the wild, it immediately leads to "copy&paste branching" (usually through multiple commits. Migrating that to Git or Hg and splitting it up can be a challenge.

Re: Monorepos: Please don’t

#94

Post by btschaegg »

adrianN wrote: post #90
0xFACEFEED wrote: post #52

At least the author gave us the courtesy of italicizing his broken assumption from the outset of the post. > Because, at scale, a monorepo must solve every problem that a polyrepo must solve, with the downside of encouraging tight coupling, and the additional herculean effort of tackling VCS scalability. Right. But you have to get to "scale" first (as it relates to VCSs). Most companies don't. Even if they're success…

You can't split monorepos after the fact, at least not without immense costs. You can always just put all your small repos into a big one.

I think there's a nuance to this that should be pointed out: Monorepos allow you to do very bad hacks (I need this other component over there; let me just put in a Symlink. Done.). And if people can, they will use those hacks.

If you split your repo up from the get go, the worst thing you can get that you'll have to assemble multiple distinct, well-encapsulated (in terms of project structure) things into one. In Git, that could lead to multiple root commits, but that's about it.

Re: Monorepos: Please don’t

#95

Post by EngineerBetter »

Visited a customer recently who had inherited a monorepo.

All their CI and release problems traced back to it.

At the risk of sounding like an old git, package coupling and package cohesion principles were defined for a reason.

I do feel like a lot of patterns in contemporary development are kneejerk reactions to how last generation's programmers did things.

Exceptions? Nah, multiple returns! Dependency management? Who needs it... Oh, wait.

Many small, single-responsibility repos? Wang it all in one, and then invent your own tooling to cope with it!

Re: Monorepos: Please don’t

#96

Post by benatkin »

sandov wrote: post #35

I agree, but also... Medium: Please don't

Agree 100%. "Pardon the interruption" followed by an article with a fixed top bar asking me to "become a member" (supporting an anti-open web tech company like it's a charity) and a fixed bottom bar asking me to sign up.

Re: Monorepos: Please don’t

#97

Post by wcdolphin »

btschaegg wrote: post #94
adrianN wrote: post #90

Earlier quoted context omitted.

You can't split monorepos after the fact, at least not without immense costs. You can always just put all your small repos into a big one.

I think there's a nuance to this that should be pointed out: Monorepos allow you to do very bad hacks (I need this other component over there; let me just put in a Symlink. Done.). And if people can, they will use those hacks. If you split your repo up from the get go, the worst thing you can get that you'll have to assemble multiple distinct, well-encapsulated (in terms of project structure) things into one. In Git,…

No. The worst case is that the engineering team spent more time working on “well encapsulated projects” than on the most important project for their business and are all now out of jobs. Most companies don’t fail because of tech debt. And certainly not because of version control tech debt.

Re: Monorepos: Please don’t

#98

Post by 0xFACEFEED »

adrianN wrote: post #90
0xFACEFEED wrote: post #52

At least the author gave us the courtesy of italicizing his broken assumption from the outset of the post. > Because, at scale, a monorepo must solve every problem that a polyrepo must solve, with the downside of encouraging tight coupling, and the additional herculean effort of tackling VCS scalability. Right. But you have to get to "scale" first (as it relates to VCSs). Most companies don't. Even if they're success…

You can't split monorepos after the fact, at least not without immense costs. You can always just put all your small repos into a big one.

> You can't split monorepos after the fact, at least not without immense costs.

Sure you can. The difficulty of doing so depends on many (many) factors. If your team does their job well then the costs won't be immense. It might be annoying, but not that hard.

Speaking in absolutes or platitudes solves nothing. Sometimes monorepos make sense. Sometimes polyrepos make sense. It's entirely dependent on what your company does.

Re: Monorepos: Please don’t

#99

Post by 0xFACEFEED »

wcdolphin wrote: post #97
btschaegg wrote: post #94

Earlier quoted context omitted.

I think there's a nuance to this that should be pointed out: Monorepos allow you to do very bad hacks (I need this other component over there; let me just put in a Symlink. Done.). And if people can, they will use those hacks. If you split your repo up from the get go, the worst thing you can get that you'll have to assemble multiple distinct, well-encapsulated (in terms of project structure) things into one. In Git,…

No. The worst case is that the engineering team spent more time working on “well encapsulated projects” than on the most important project for their business and are all now out of jobs. Most companies don’t fail because of tech debt. And certainly not because of version control tech debt.

Exactly. Whenever I see an engineer take a hardline position (eg: "no monorepos you zealots!") I always ask myself: is this person just annoyed?

Most of the time they're just annoyed.

One side effect of every successful business are annoyed worker ants that are sick of dealing with growth problems. I've been there. I know how annoying it can be.

Personally I've found comfort in embracing the chaos and learning to manage it responsibly. No dogma. No absolutes. Know how to do monorepos well. Know how to do polyrepos well. Learn the pitfalls of both. Don't assume other people are stupid zealots.

Re: Monorepos: Please don’t

#100

Post by nathan_f77 »

I'm using a monorepo as a solo developer, and it's been pretty good. I like having everything in one place, so I can work on everything in a branch, including the feature, updates to API clients, documentation, blog post, etc.

One problem is that my test suite is very inefficient. I have to run through every integration test, even if I haven't changed any code that might cause these tests to fail. It's especially weird that CI runs all my tests whenever I write a new blog post. So I'm very tempted to split up some things into internal libraries and keep them in a separate repo, and add all these repos as submodules. I know this can be pretty dangerous, and it's easy to break things when you update dependencies, OS versions, language versions, etc.

If I go down this road, I have to be extremely careful to enumerate all the things that might break the library, and prevent any of these things from being updated automatically. I'll set a very strict version constraint in the package.json / gemspec, and throw an error if I detect a different version of Node, Python, Ruby, system libraries, etc. Then I'm forced to run all the library tests and explicitly bump the versions if I want to update anything.

I should also only do this when the library is a pure function with no side effects.

The really tricky part is figuring out how to write robust integration tests. API boundaries can be a big source of bugs. I think I'll do something similar to VCR [1], where the first integration test executes all of the code without any mocks, and then records the response. The response would then include those exact arguments, and it would also be tied to a specific commit hash for the library. If I change anything in the library, then I just need to re-run the slow tests, and then everything will be cached. I guess a real advantage of putting things in a separate library is that you know exactly what files are required for a specific feature, and the commit hash gives you a "fingerprint" of those files that you can use for caching in your tests.

Just have to be super careful about any dependencies that might break the library. Also I really need to start running all my tests in a Docker container which matches CI and production. I even have some screenshot tests where I have alternative versions for Mac and Linux. Would be nice to delete those. The experience was really bad when I tried to do this in the past, so I need to figure out a better way.

Anyway, sorry for the train of thought! Would be interested to hear your thoughts, and if there's anything else I should watch out for.

[1] https://github.com/vcr/vcr

Post reply on HN