Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Debian Git Monorepo

blog.liw.fi

Post reply on HN
Print view View original

31–40 of 157 posts

Re: Debian Git Monorepo

#31

Post by BlackFly »

Cu3PO42 wrote: post #16
G3rn0ti wrote: post #7

I know it‘s an April‘s fool. But I don’t get this recent trend of people arguing in favor of a monorepo (at my work place, too). It’s a nightmare to handle and use. What gives?

There are real, tangible benefits to monorepos. And real, tangible downsides, too. Like everything else, it's a tradeoff. I'm sure people advocating for them have brought up some benefits. But in case they haven't, here's an example I recently encountered at work: for an application we are developing, we needed functionality X. We knew that we'd need X in other projects, too, so we made it a library. I published it t…

The real problems with monorepos are that most of the benefits vanish as the scale increases unless you invest into building more monorepo tooling.

In your particular case, if your library becomes too popular, your one or two line implementation detail changes ripple out and trigger rebuilds of too many downstreams, many of which will have flakey tests and fail your MR. If most users are not actually depending on that functionality, or if you simply are doing semver properly, then you could avoid rebuilding those dependencies. Eventually the builds take too much time and CI rejects the pipelines or you continuously bump the build timeout but wait longer and longer for your changes to go live. You can solve these problems, if you invest in more monorepo tooling.

Similarly, once you are too popular a library in a monorepo, you will never do any atomic breaking API changes since it would require updating too many downstreams. Instead you will fake version it: add a new API, migrate users to it in multiple commits, delete the old version. Some of these migrations run out of steam midway in the biggest phase: phase 2. This approach does have the benefit of forcing the upstream author to make the two versions of the API co-exist.

Of course I am talking about scales where real limits start to break down. When your codebase is larger than your ram, an index won't fit into RAM anymore and every code search requires disk or network I/O. Eventually your repository doesn't fit on disk anymore and you interact with it with a special IDE and only download things you start editing or perform sparse checkouts in the first place so discoverability is again a problem.

Edit: of course some problems crop up sooner than hard limits are reached, like the flakey test issue I mentioned as well as visibility and control of changes to actual maintainers.

Re: Debian Git Monorepo

#32

Post by surajrmal »

Archelaos wrote: post #18
Cu3PO42 wrote: post #16

Earlier quoted context omitted.

There are real, tangible benefits to monorepos. And real, tangible downsides, too. Like everything else, it's a tradeoff. I'm sure people advocating for them have brought up some benefits. But in case they haven't, here's an example I recently encountered at work: for an application we are developing, we needed functionality X. We knew that we'd need X in other projects, too, so we made it a library. I published it t…

> Turns out, feature X wasn't as well understood as we thought it was and we are still changing things around all the time. If you publish something as a library for someone else, you should generally commit to supporting the public interface of that library as if it were a long-term support release. Exactly how long that should be depends on many factors, of course. But as a rule of thumb, I would recommend supporti…

This guidance is far too specific to make sense in the general case. Not every scenario has the same constraints. If I had to pay a 2 year maintenance cost for every library I write and wanted to reuse between projects I'd never end up writing it. If you can migrate all library clients over to the new version, there is no reason to maintain old versions.

Re: Debian Git Monorepo

#33

Post by ownagefool »

Archelaos wrote: post #18
Cu3PO42 wrote: post #16

Earlier quoted context omitted.

There are real, tangible benefits to monorepos. And real, tangible downsides, too. Like everything else, it's a tradeoff. I'm sure people advocating for them have brought up some benefits. But in case they haven't, here's an example I recently encountered at work: for an application we are developing, we needed functionality X. We knew that we'd need X in other projects, too, so we made it a library. I published it t…

> Turns out, feature X wasn't as well understood as we thought it was and we are still changing things around all the time. If you publish something as a library for someone else, you should generally commit to supporting the public interface of that library as if it were a long-term support release. Exactly how long that should be depends on many factors, of course. But as a rule of thumb, I would recommend supporti…

It depends.

If you drop a library for internal usage only and you want to change the contract, and the tests, library, and api are all in the same repo, you just change them in a single PR and that's done. This works because the single commit hash contains all information. It requires people put their code in the monorepo and hookup their tests, but assuming they do, you can build a reasonable degree of confidence on a green build.

Once you separate the library different repo, and let people consume that as something in their own repos, you need to do the versioning dance as you have no idea if their still is still working.

A lot of people go for the latter because it contextually allows them to ignore the rest of the stack, and there are some pros to doing this, but testing, deploying, versioning, etc, all become more difficult, and that's something struggle with.

Thus, unless you have a crap load of code / commits, monorepos arguably have more advantages than disadvantages.

Re: Debian Git Monorepo

#34

Post by daghamm »

nolist_policy wrote: post #3

April fool aside, what I really want is a Debian git monorepo of submodules, where each submodule points to the upstream git repository.

What do you think of the openbsd model? It has a mono repo for the os and a ports repo for everything else

(I think)

Re: Debian Git Monorepo

#35

Post by actionfromafar »

Archelaos wrote: post #18
Cu3PO42 wrote: post #16

Earlier quoted context omitted.

There are real, tangible benefits to monorepos. And real, tangible downsides, too. Like everything else, it's a tradeoff. I'm sure people advocating for them have brought up some benefits. But in case they haven't, here's an example I recently encountered at work: for an application we are developing, we needed functionality X. We knew that we'd need X in other projects, too, so we made it a library. I published it t…

> Turns out, feature X wasn't as well understood as we thought it was and we are still changing things around all the time. If you publish something as a library for someone else, you should generally commit to supporting the public interface of that library as if it were a long-term support release. Exactly how long that should be depends on many factors, of course. But as a rule of thumb, I would recommend supporti…

This is orthogonal to mono-repo vs not mono-repo and indeed distribution mechanism.

Re: Debian Git Monorepo

#36

Post by actionfromafar »

Repulsion9513 wrote: post #27
The_Colonel wrote: post #24

Earlier quoted context omitted.

This is exactly the friction/overhead the other person means and what monorepo works around. (but also keep in mind this is an internal library for likely a couple of internal apps, supporting BC for 2 years is in most cases unwarranted)

Monorepo doesn't "work around" it, it means you have to support everything (including migrating/BC) on the current version, instead of having to support old stuff on the old version.

You can “fork” inside the monorepo too, just take a copy.

Re: Debian Git Monorepo

#37

Post by surajrmal »

G3rn0ti wrote: post #7

I know it‘s an April‘s fool. But I don’t get this recent trend of people arguing in favor of a monorepo (at my work place, too). It’s a nightmare to handle and use. What gives?

I struggle to understand the upside of avoiding monorepos. A lot of people think it means you can't decouple releases, modularize builds and similar with a monorepo, but that's simply not true. The efficiency at which you can refactor and upgrade your codebase cannot be understated.

Re: Debian Git Monorepo

#38

Post by actionfromafar »

fragmede wrote: post #23
baq wrote: post #11

Earlier quoted context omitted.

a hundred small repos are a different kind of nightmare. the truth is, having a large code base is just hard no matter which way you handle it. you'll end up with custom repo tooling for the monorepo or blown up CI/CD infrastructure for many small repos either way. complexity will be conserved; it can be transferred, but can't be removed.

baq's law of code thermo dynamics: complexity cannot be removed. it can be transferred but can not be removed. your CI/CD system will be as complicated as your repo structure isn't.

Saving

Re: Debian Git Monorepo

#39

Post by Archelaos »

surajrmal wrote: post #32
Archelaos wrote: post #18

Earlier quoted context omitted.

> Turns out, feature X wasn't as well understood as we thought it was and we are still changing things around all the time. If you publish something as a library for someone else, you should generally commit to supporting the public interface of that library as if it were a long-term support release. Exactly how long that should be depends on many factors, of course. But as a rule of thumb, I would recommend supporti…

This guidance is far too specific to make sense in the general case. Not every scenario has the same constraints. If I had to pay a 2 year maintenance cost for every library I write and wanted to reuse between projects I'd never end up writing it. If you can migrate all library clients over to the new version, there is no reason to maintain old versions.

I would not call this a "library", but a set of interrelated project with "shared moduls" instead. With a "library" I mean a piece of code where the publisher does know who is using it for what exact purpose. Of couse we could now fight over a proper general definition for "library", but this is how I would use the term in this context, because thee long term aspect is what really makes outsourcing such code meaningful -- in contrast to the parents observation that it does not make sense to incorporate foreign code that constantly changes.

Re: Debian Git Monorepo

#40

Post by Xeamek »

G3rn0ti wrote: post #7

I know it‘s an April‘s fool. But I don’t get this recent trend of people arguing in favor of a monorepo (at my work place, too). It’s a nightmare to handle and use. What gives?

> It’s a nightmare to handle and use.

Why? If all Your work and knowledge is confined in one repo, there still is a way to work only on that part. Or at least there should be.

But if You want/need to have access to other parts, it's much easier to use monorepo then to search trough 100 of github repos that are somehow dependent but it's not obvious in what way.

Post reply on HN