Monorepos: Please don’t • Hacker News Board

Re: Monorepos: Please don’t

#341

Post by joshuamorton » Thu, Jan 03, 2019, 5:37 PM UTC

tigershark wrote: post #319

ryanbrunner wrote: post #214

Earlier quoted context omitted.

In cases where there's a high degree of churn (i.e. early-stage startups) in shared libraries, updating those libraries can cause a large amount of busywork and ceremony. If you had a `foo()` function shared between the GUI and the server (or two services on your backend, or whatever), in a monorepo your workflow is: - Update foo() - Merge to master - Deploy In a polyrepo where foo() is defined in a versioned, shared…

It seems to me that the real problem is your toolchain. In a previous project the workflow was like this: Update foo() Merge to master Publish shared library Deploy So as you can see the only step added was to publish the shared library that would automatically update the version in all the projects using it. If you are really doing everything manually I can understand that this is a pain, but this has nothing to do…

But you've just invented a sharded monorepo, and now have all the monorepo problems without the solutions.

What if updating foo() breaks something in one of the clients (say due to reliance on something not specified). Then you didn't catch that issue by running client's tests, now client is broken, and they don't necessarily know why. They know the most recent version of shared broke them, but then they have to say "you broke me" or now one of the teams needs to investigate and possibly needs to bisect across all changes in the version bump under their tests to find the breakage.

How is that handled?

(the broader point here is that monorepo or multirepo is an interface, not an implementation, its all a tooling problem. There are features you want your repo to support. Do you invest that tooling in scaling a single repo or in coordinating multiple ones? Maybe I should write that blog post).

Re: Monorepos: Please don’t

#342

Post by manigandham » Thu, Jan 03, 2019, 5:50 PM UTC

bpicolo wrote: post #221

ajuc wrote: post #31

I call bullshit on "our repository is too big for one machine". Seriously, you have over 1 TB of code and 100 people wrote it?

In 2016, Google's monorepo was 86TB

The keyword there is "Google". Everything is different at the extremes.

Re: Monorepos: Please don’t

#343

Post by lsferreira42 » Thu, Jan 03, 2019, 5:51 PM UTC

yowlingcat wrote: post #256

I think this article is complete horseshit. A monorepo will serve you 99% of the time until you hit a certain level of scale when you get to worry about whether a monorepo or a polyrepo is actually material. Most cases are never going to get there. Before that point, a polyrepo is purely a distraction and makes synchronous deployment really painful. We had to migrate a polyrepo to a monorepo and it was not fun becaus…

This article is too agressive and have a childish language that is not for my taste.

Re: Monorepos: Please don’t

#344

Post by manigandham » Thu, Jan 03, 2019, 5:58 PM UTC

As always, the real world is a whole lot of gray between the black and white articles that are fun but useless.

Multirepos like microservices are all about scaling people, not the project. Start with the monolith and monorepo until you need to split, and then focus on separating by groups of logical functionality or team responsibilities (although if those are different then you'll end up with other problems).

Also stop taking things literally. Monorepo does not mean you must have everything in a single repo. Even a startup can put the majority of the codebase in one place and have things like a corporate website or small admin backend in another.

Re: Monorepos: Please don’t

#345

Post by Chris2048 » Thu, Jan 03, 2019, 6:00 PM UTC

laumars wrote: post #189

tigershark wrote: post #179

Earlier quoted context omitted.

Because obviously the changes that you make in the gui are completely isolated from the changes you make on the server. When you are working on the gui the server code is just noise and vice versa. And it gets even worse when you use two different languages for the gui and the server.

That only really matters if your backend developers are a different team to your frontend developers where they'd want to be working concurrently. And even then, they could work in different branches and both teams merge into a development branch when finished. The idealistic discussions for or against monorepos often overlook the most important detail: who's working on the code and how would you want them to version…

> That only really matters if your backend developers are a different team to your frontend developers

What if there is one team, but different developers (one working on the frontend, another on the back)? What if QA can test the API while the frontend development is ongoing?

What if the front and backends have different toolchains, and ultimately separate execution environments (server app backend vs JS running on client machines).

Re: Monorepos: Please don’t

#346

Post by KuhlMensch » Thu, Jan 03, 2019, 6:07 PM UTC

Hm, I find monorepos are a natural in javascript land. There is allot less wiring afforded by a little meta-orchestration. This is especially helpful in the repos I've worked on.

But from reading the article, it seems like there are legitimate areas where they might not fit.

Re: Monorepos: Please don’t

#347

Post by userulluipeste » Thu, Jan 03, 2019, 6:12 PM UTC

This is just modularity in the broader information-related development handling. People tend to get political about things that make their involvement comfortable and/or when it's not them who are to deal with the pesky consequences. I strongly suspect that that must have been the cause for how giants like Google¹ or Facebook² ended up with monorepos. It is developers' workout or letting them be with their cake; kick the problem down the road and hope to acquire in time the resources necessary to throw at it later.

¹² "Don't be evil" (with developers, among others) and "move fast and break things" most definitely asks for cutting a few corners here and there.

Re: Monorepos: Please don’t

#348

Post by drugme » Thu, Jan 03, 2019, 6:35 PM UTC

yowlingcat wrote: post #269

drugme wrote: post #264

Earlier quoted context omitted.

I found it to be neither horseshit nor irresponsible. A bit overdrawn and skewed in some of its arguments, perhaps. But then again... so was your critique. For example: We had to migrate a polyrepo to a monorepo and it was not fun because it was a migration that should have never had to be done in the first place s/polyrepo/monorepo/ in the above and you have an assertion of about equal plausibility and weight.

No, it is horseshit. 99% of companies will never hit big company VCS scaling issues, and once they do, they're on their own. To characterize that scale as common is one of the most embarrassing failures of modern software engineering. People are so embarrassed to use well worn tooling and accept that large scale is both uncommon and something that doesn't invalidate tried and true patterns for smaller scales. It's ut…

No, it is horseshit [because scale]

The thing is, scale was only one factor listed among many.

Re: Monorepos: Please don’t

#349

Post by diminoten » Thu, Jan 03, 2019, 6:38 PM UTC

rakoo wrote: post #324

diminoten wrote: post #237

Earlier quoted context omitted.

Easier because you can commit more atrocities. Easier is not better, some things should be hard to do to dissuade you from doing them. Stop burdening your co-workers! People in here keep saying "easy" like it's the end goal, but it's not. Correct is. Writing great software is hard, and monorepos make it even harder to do that because a monorepos encourages an "anything goes" vibe.

I don't understand how you can say that monorepos and correct cannot coexist. Don't you have the minimum that is code quality analysis, automated testing and mandatory code reviews in place? Those must exist to maximize correctness, and they must exist wheter you have 1 or 100 repos. If I can commit more atrocities I can also commit more fine things, and I have infra in place to stop crap making it to master. Like th…

I didn't say cannot, I said that correctness in a monorepo exists despite the monorepo, not because of the monorepo.

Re: Monorepos: Please don’t

#350

Post by brokentone » Thu, Jan 03, 2019, 6:44 PM UTC

curtis wrote: post #317

thanatos_dem wrote: post #289

Earlier quoted context omitted.

I work on CI/CD systems, and that’s one thing that definitely gets harder in a monorepo. So you made a commit. What artifacts change as a result? What do you need to rebuild, retest, and redeploy? It doesn’t take a large amount of scale to make rebuilding and retesting everything impossible. In a poly repo world, the repository is generally the unit of building and deployment. In monorepo it gets more messy. For inst…

> So you made a commit. What artifacts change as a result? What do you need to rebuild, retest, and redeploy? If you're using Git, then typically for each push to the remote repository you get a notification with this data in it: BRANCH # the remote branch getting updated OLD_COMMIT # the commit the branch ref was pointing to before the push NEW_COMMIT # the commit the branch ref was pointing to after the push # To g…

This is something I've been working on a bit myself.

Figuring out which files change is relatively easy (as you've demonstrated). Figuring out what the impact of that is quite hard in non-compiled languages (tools like Maven, Buck, Bazel, etc do this well for compiled languages). I.e. In a repo which is primarily JavaScript, I can get the list of changed files, and hopefully have unit test files which are obviously linear to those. However, knowing if these are depended on by other files/modules (at some depth) is much harder. Same for integration tests -- which of these are related?