Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Monorepos: Please don’t

medium.com

Post reply on HN
Print view View original

281–290 of 402 posts

Re: Monorepos: Please don’t

#281

Post by etxm »

I feel like a lot of arguments against monorepos assume micro services are the _only_ option on the other side.

I tend to break up most of my projects at the edge of business logic or domain logic and lean on a package manager to “deploy together” like any other dependency that’s not in your repo.

This allows teams to work independently without a large sprawling repo. If you’re following anything semver-ish hopefully your other teams in the company aren’t breaking releases and you can auto-upgrade on patch level changes. If not, we’ll thank goodness CI is there.

I’ve always had difficulty working in projects with too many purposes. This helps me focus where to put things and gives an easy point of escape if a dependency needs to become a service in the future.

Re: Monorepos: Please don’t

#282

Post by Eridrus »

bborud wrote: post #262
rossjudson wrote: post #148

Hilariously misguided. Pretty funny to read that the things I do every day are impossible. Monorepo and tight coupling are orthogonal issues. Limits on coupling come from the build system, not from the source repository. Yes, you should assume there is a sophisticated "VFS". What is this "checkout" you speak of? I have no time for that. I am too busy grepping the entire code base, which is apparently not possible. If…

It can work. That doesn't mean it is a universal solution. And it doesn't even mean it is a solution that is guaranteed to cover most projects. Whether or not a monorepo works depends on a lot of factors. In my experience the number of cases where it doesn't work appears to outnumber the cases where it works. It can work nicely when you have disciplined and demonstrably above average programmers that are good at stru…

At some point blaze added the concept of visibility to control dependencies, so teams can whitelist users of their code. Though you can always comment it out while developing locally.

Re: Monorepos: Please don’t

#283

Post by mlthoughts2018 »

yowlingcat wrote: post #269
drugme wrote: post #264

Earlier quoted context omitted.

I found it to be neither horseshit nor irresponsible. A bit overdrawn and skewed in some of its arguments, perhaps. But then again... so was your critique. For example: We had to migrate a polyrepo to a monorepo and it was not fun because it was a migration that should have never had to be done in the first place s/polyrepo/monorepo/ in the above and you have an assertion of about equal plausibility and weight.

No, it is horseshit. 99% of companies will never hit big company VCS scaling issues, and once they do, they're on their own. To characterize that scale as common is one of the most embarrassing failures of modern software engineering. People are so embarrassed to use well worn tooling and accept that large scale is both uncommon and something that doesn't invalidate tried and true patterns for smaller scales. It's ut…

In my experience, monorepos cause outrageous problems that have nothing to do with scale. Small or medium monorepos are equally as terrifying.

It’s much more about coupling and engendering reliance on pre-existing CI constraints, pipeline constraints, etc. If you work in a monorepo set up to assume a certain model of CI and delivery, but you need to innovate a new project that requires a totally different way to approach it, the monorepo kills you.

Another unappreciated problem of monorepos is how they engender monopolicies as well, and humans whose jobs become valuable because of their strict adherence to the single accepted way of doing anything will, naturally, become irrationally resistant to changes that could possibly undermine that.

It’s a snowball effect, and often the veteran engineers who have survived the scars of the monorepo for a while will be the biggest cheerleaders for it, like some type of Stockholm syndrome, continually misleading management by telling them the monorepo can always keep growing by attrition and will be fine and keep solving every problem, unto the point that it starts breaking in collossal failures and people are sitting around confused why some startup is eating their lunch and capable of much faster innovation cycles.

Re: Monorepos: Please don’t

#284

Post by jen20 »

fnord123 wrote: post #244
EngineerBetter wrote: post #95

Visited a customer recently who had inherited a monorepo. All their CI and release problems traced back to it. At the risk of sounding like an old git, package coupling and package cohesion principles were defined for a reason. I do feel like a lot of patterns in contemporary development are kneejerk reactions to how last generation's programmers did things. Exceptions? Nah, multiple returns! Dependency management? W…

>Exceptions? Nah, multiple returns! Dependency management? Who needs it... Oh, wait. I thought consensus was that exceptions, like OOP, is an antipattern. I guess there's room for different opinions. :-/

I would not assume there is _any_ consensus at all that object orientation (as defined by Kay, not by C++) is an anti-pattern.

Re: Monorepos: Please don’t

#286

Post by swiftcoder »

oblio wrote: post #174
nindalf wrote: post #171

Earlier quoted context omitted.

This makes sense to me. If you're considering a monorepo with millions of lines of code used concurrently by thousands of developers, it's absolutely possible. You just need a handful of developers working on the infra to make it happen. I do agree with GP though. I wish the author hadn't decided the things I do everyday are impossible.

> You just need a handful of developers working on the infra to make it happen. With thousands of developers banging on the code base, it's going to be more than "a handful of developers". It's going to be at least a few "handfuls" of developers full time plus probably many, many other full time equivalents spread out throughout the whole user base (testing, supporting other users, etc.).

In reality, it's several hundred developers working full time on infra, and they are all overworked, and gradually falling behind. Monorepos at the scale of Google/Facebook are hard.

We aren't talking about maintaining Mercurial here - we are talking about developing a brand new distributed VCS that happens to be 'Mercurial-compatible', and deploying/maintaining it for tens of thousands of developers working simultaneously.

Re: Monorepos: Please don’t

#287

Post by sangnoir »

yowlingcat wrote: post #269
drugme wrote: post #264

Earlier quoted context omitted.

I found it to be neither horseshit nor irresponsible. A bit overdrawn and skewed in some of its arguments, perhaps. But then again... so was your critique. For example: We had to migrate a polyrepo to a monorepo and it was not fun because it was a migration that should have never had to be done in the first place s/polyrepo/monorepo/ in the above and you have an assertion of about equal plausibility and weight.

No, it is horseshit. 99% of companies will never hit big company VCS scaling issues, and once they do, they're on their own. To characterize that scale as common is one of the most embarrassing failures of modern software engineering. People are so embarrassed to use well worn tooling and accept that large scale is both uncommon and something that doesn't invalidate tried and true patterns for smaller scales. It's ut…

> It's utterly baffling to me.

It's not hard to explain: Scale has been fetishized by the industry/trade. Everyone wants the cachet of working at scale. 1.5 GB of CSV text? That's Big Data, let's break out map-reduce. 1 load balancer and not enough servers to fill half a rack? That's a scalable architecture, we could scale to multiple datacenters at some point in the future, so let's design it now.

Deploying oversized solutions is partly due to outsiders jonesing the scale of Google, Fb and gang, partly Resume-stuffing ("I have worked with this tech before"), and lastly FANG diasporans who miss the tech they used and rewrite systems/evangelize the effectiveness of those solutions to much smaller organizations.

Re: Monorepos: Please don’t

#288

Post by artemave »

bad_user wrote: post #109
curtis wrote: post #32

My advice is that if components need to release together, then they ought to be in the same repo. I'd probably go further and say that if you just think components might need to release together then they should go in the same repo, because you can in fact pretty easily manage projects with different release schedules from the same repo if you really need to. On the other hand if you've got a whole bunch of component…

Components might need to be released “together”, but if they are worked on by different teams, it means they’ll have a different release process, as in different timeline, different priorities. First of all this is normal, because otherwise the development doesn’t scale. In such a case the monorepo starts to suck. And that’s the problem with your philosophy ... it matters less how the components connect, it matters m…

> Having different teams working on different components in different repos will lead to an interesting effect ... nobody wants to own more than they have to

... and so nobody really understands how all of the components tie together and as a result it takes weeks of manual testing to release.

Re: Monorepos: Please don’t

#289

Post by thanatos_dem »

yowlingcat wrote: post #256

I think this article is complete horseshit. A monorepo will serve you 99% of the time until you hit a certain level of scale when you get to worry about whether a monorepo or a polyrepo is actually material. Most cases are never going to get there. Before that point, a polyrepo is purely a distraction and makes synchronous deployment really painful. We had to migrate a polyrepo to a monorepo and it was not fun becaus…

I work on CI/CD systems, and that’s one thing that definitely gets harder in a monorepo.

So you made a commit. What artifacts change as a result? What do you need to rebuild, retest, and redeploy? It doesn’t take a large amount of scale to make rebuilding and retesting everything impossible. In a poly repo world, the repository is generally the unit of building and deployment. In monorepo it gets more messy.

For instance, one perceived benefit of a monorepo is it removes the need for explicit versioning between libraries and the code that uses them, since they’re all versioned together.

But now, if someone changes the library, you need to have a way to find all of its usages, and retest those to make sure the changed didn’t break their use. So there’s a dependency tree of components somewhere that needs to be established, but now it’s not explicit, and no one is given the option to pin to a particular version if they can’t/won’t update. This is the world of google & influcenced the (lack of) dependency management in go.

You could very well publish everything independently, using semver, and put build descriptors inside each project subdirectory, but then, congratulations, you just invented the polyrepo, or an approximation thereof.

Re: Monorepos: Please don’t

#290

Post by zeveb »

sangnoir wrote: post #287
yowlingcat wrote: post #269

Earlier quoted context omitted.

No, it is horseshit. 99% of companies will never hit big company VCS scaling issues, and once they do, they're on their own. To characterize that scale as common is one of the most embarrassing failures of modern software engineering. People are so embarrassed to use well worn tooling and accept that large scale is both uncommon and something that doesn't invalidate tried and true patterns for smaller scales. It's ut…

> It's utterly baffling to me. It's not hard to explain: Scale has been fetishized by the industry/trade. Everyone wants the cachet of working at scale . 1.5 GB of CSV text? That's Big Data , let's break out map-reduce. 1 load balancer and not enough servers to fill half a rack? That's a scalable architecture, we could scale to multiple datacenters at some point in the future, so let's design it now. Deploying oversi…

To be fair, part of the problem is that each of us has been bitten throughout his career by issues which could have been prevented by being able to predict the future. We then move from the truth that if we had known the future, we could have acted better yesterday to the fallacy that today we finally know what we're going to need tomorrow.

This isn't isolated to our industry, of course: a constant refrain is that generals & admirals fight the last war; the financial industry is rife with products which are secure against the last recession, and so forth.

Post reply on HN