Monorepos: Please don’t • Hacker News Board

Re: Monorepos: Please don’t

#241

Post by thedufer » Thu, Jan 03, 2019, 1:52 PM UTC

Maybe its just that the author's cutoff is at the wrong team size, but the monorepo I work on (with ~150 devs) has almost none of the problems presented.

Unreasonable for a single dev to have the entire repo? I'm looking at a repo with ~10 million LoC and ~1.4 million commits. I have 74 different branches checked out right now. Hard drives are cheap.

Code refactors are impossible? I reviewed two of those this morning. They're essentially a non-event. I'm not sure what to make of the merge issue - does code review have to start over after a merge? That seems like a deep issue in your code review process. The service-oriented point seems like a non-sequitur, unless you're telling me I'm supposed to have a service for, say, my queue implementation or time library.

The VCS scalability issue is the only real downside I see here. And it is real, but it also seems worth it. It helps that the big players are paving the way here - Facebook's contributions to the scalability of mercurial has definitely made a difference for us.

Re: Monorepos: Please don’t

#242

Post by wsy » Thu, Jan 03, 2019, 1:54 PM UTC

btschaegg wrote: post #127

wcdolphin wrote: post #97

Earlier quoted context omitted.

No. The worst case is that the engineering team spent more time working on “well encapsulated projects” than on the most important project for their business and are all now out of jobs. Most companies don’t fail because of tech debt. And certainly not because of version control tech debt.

Not exactly. (At least small) companies can go out of business because of bugs. And one great way to "achieve" said bugs are implicit dependencies hidden from developers that didn't introduce them. > The worst case is that the engineering team spent more time working on “well encapsulated projects” than on the most important project for their business I'm not really sure how I should read this. Don't you use your rep…

If you do a poly-repo approach from the start, and have dependencies between repos, you need to introduce component versioning from the start. Component versioning doesn't solve any business problems, but requires engineering effort.

Re: Monorepos: Please don’t

#243

Post by hvindin » Thu, Jan 03, 2019, 1:59 PM UTC

I suspect the problem most people end up trying to solve isn't "how do I technically scale my tools", because the author points out that tools and techniques for this already exist and its an already solved problem.

Instead my experience has largely been that the problem to solve is "how do I make some few hundred developers behave in a predictable way" in the scenario where you have many ok developers, but that you can't really be sure that none of them will break stuff because you are trying to solve organisational problems of keeping people with merge rights to only the people who wont break things but at the same time not bottle necking development on to small a number of people then sure, split your repos up so that people can only break stuff that they 'own'.

But at least be honest about the fact that most of the technical issues of having a monorepo have been solved already so the issues you are probably trying to solve are actually people problems.

Re: Monorepos: Please don’t

#244

Post by fnord123 » Thu, Jan 03, 2019, 2:11 PM UTC

EngineerBetter wrote: post #95

Visited a customer recently who had inherited a monorepo. All their CI and release problems traced back to it. At the risk of sounding like an old git, package coupling and package cohesion principles were defined for a reason. I do feel like a lot of patterns in contemporary development are kneejerk reactions to how last generation's programmers did things. Exceptions? Nah, multiple returns! Dependency management? W…

>Exceptions? Nah, multiple returns! Dependency management? Who needs it... Oh, wait.

I thought consensus was that exceptions, like OOP, is an antipattern. I guess there's room for different opinions. :-/

Re: Monorepos: Please don’t

#245

Post by rbetts » Thu, Jan 03, 2019, 2:11 PM UTC

madhadron wrote: post #73

rbetts wrote: post #30

Unless you are pure OSS or pure closed source - you end up with a poly-repo strategy regardless as you split open and closed code, suffering the annoyances of both systems.

No, what you end up with is a system for mirroring open source code into your repo, and a system for mirroring commits that should be open source from your code into external repos. All active work still happens in a monorepo.

Any good examples of small companies that pull this off and the bots/CI/tool whatever they use to do it?

Re: Monorepos: Please don’t

#246

Post by zbentley » Thu, Jan 03, 2019, 2:14 PM UTC

The VCS/codebase-tooling-size argument rings a bit hollow.

We have really good code-search tools that are heavily optimized and indexed (from ripgrep/silversearcher to more centralized things like hound, when local-disk performance just won't cutit).

It's not hard to optimize Git workflows to be faster with relatively simple tricks, and if that absolutely doesn't scale for some reason and VFS isn't an option, there are always centralized VCS systems like Perforce that solve this. P4 gets a lot of shit, but it's really good at solving the gigantic-repo domain; tune your client properly and you can initial-sync 10+ GB repos in the time it takes to get a cup of coffee (and, if your company is large/old enough to have a repo that big, it can probably afford the Perforce licenses).

Re: Monorepos: Please don’t

#247

Post by qwerty456127 » Thu, Jan 03, 2019, 2:18 PM UTC

I didn't actually know somebody already does this, but a conceptual idea has came into my mind yesterday: what if there was just one big code repository for a particular programming language and everything anybody writes would immediately become a part of the standard library? It feels kind of a collective brain...

Re: Monorepos: Please don’t

#248

Post by zbentley » Thu, Jan 03, 2019, 2:18 PM UTC

outsomnia wrote: post #91

0xFACEFEED wrote: post #52

At least the author gave us the courtesy of italicizing his broken assumption from the outset of the post. > Because, at scale, a monorepo must solve every problem that a polyrepo must solve, with the downside of encouraging tight coupling, and the additional herculean effort of tackling VCS scalability. Right. But you have to get to "scale" first (as it relates to VCSs). Most companies don't. Even if they're success…

The author talks about proponents of monorepos, but I thought when I read it: actually they are victims of monorepos trying to explain to themselves as much as anyone why they choose to suffer with them. (Actual reason: for $$$). Nobody would choose to drag around every historical afterthought in the development sequence of long forgotten software going back three decades that no longer builds with current tools, jus…

Bloat and the failure to remove broken/bad legacy code have nothing to do with monorepos.

"Hey, what does this server do?"

"No idea; it hasn't been touched in years. What's deployed to it?"

"Some 'foobar-ng' thing, never heard of that. Says it was last updated 5 years ago. Pull up our source repo for that package, will you?"

"Hang on, we've got like 30 services with names containing 'foobar', let me find that one . . . oh god. You don't wanna know."

"Fuck it, I'm just gonna shut this server down and remove that ancient, dead, busted package."

"The main billing system just broke! What did you do?!"

Re: Monorepos: Please don’t

#249

Post by icedchai » Thu, Jan 03, 2019, 2:19 PM UTC

Monorepos are way simpler for small teams to work with. At my startup we have roughly 10 services out of the same repo. It's much easier to "cut a release" across the entire system. It's much easier to share code internally, upgrade dependencies, etc.

For a larger company, it might not be a good idea. However, most startups start small and stay that way. Why take on the overhead you don't need?

Re: Monorepos: Please don’t

#250

Post by marcinzm » Thu, Jan 03, 2019, 2:28 PM UTC

outsomnia wrote: post #91

0xFACEFEED wrote: post #52

At least the author gave us the courtesy of italicizing his broken assumption from the outset of the post. > Because, at scale, a monorepo must solve every problem that a polyrepo must solve, with the downside of encouraging tight coupling, and the additional herculean effort of tackling VCS scalability. Right. But you have to get to "scale" first (as it relates to VCSs). Most companies don't. Even if they're success…

The author talks about proponents of monorepos, but I thought when I read it: actually they are victims of monorepos trying to explain to themselves as much as anyone why they choose to suffer with them. (Actual reason: for $$$). Nobody would choose to drag around every historical afterthought in the development sequence of long forgotten software going back three decades that no longer builds with current tools, jus…

>Nobody would choose to drag around every historical afterthought in the development sequence of long forgotten software going back three decades that no longer builds with current tools, just so they can work on a small library off in a corner.

If it doesn't work then it should be deleted. If it's still running somewhere then it should be maintained. Presumably you have a CI system so the monorepo actually requires everything in it to build.

In my experience, it's polyrepos that allow for dead and un-maintained code to just sit there for eternity. You forget about that unused repo right until the moment the service it deploys to (if you can track down that dependency) needs an update or goes down. Monorepos can more easily force system wide CI that checks for broken dependencies or other issues.