Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Monorepoize – Bash scripts for creating a monorepo out of smaller repos

github.com

Post reply on HN
Print view View original

11–20 of 103 posts

Re: Monorepoize – Bash scripts for creating a monorepo out of smaller repos

#11

Post by cryptica »

stepbeek wrote: post #10
edwintorok wrote: post #6

Earlier quoted context omitted.

Once your code grows and you end up with too many repos you'll be yearning to get back to monorepos. It allows to make changes in one go instead of having 3-4 PRs that all have to be merged at same time or otherwise the build breaks. Having separate repos only makes sense if they are maintained by separate teams with a well defined API in-between that doesn't change often. If the same team maintains all repos then it…

> Having separate repos only makes sense if they are maintained by separate teams with a well defined API Agree! I've found that a multirepo setup only works if there's an obvious boundary. Similar to SOA, distributed monoliths - or distributed monorepos - are problematic.

> Having separate repos only makes sense if they are maintained by separate teams with a well defined API

If some logic is modular then it should be in a separate repo. If it's not modular then it should be part of the same repo; just regular files inside directories... IMO, monorepos try to have it both ways but this doesn't make any sense; it encourages developers to write logic which is "somewhat modular".

But "somewhat modular" is not modular at all. Either it's modular and it can easily be extracted out completely or it's not and then it should be embedded into the project's code.

Code which is not modular should never need to be shared with other projects.

Re: Monorepoize – Bash scripts for creating a monorepo out of smaller repos

#12

Post by unixhero »

beagle3 wrote: post #9
cryptica wrote: post #7

Big corporations are extremely inefficient. Why does everyone want to copy them? And why do we use all the bulky tools they produce when there are far simpler, better and more open alternatives available?

Because it turns out big corporations that use multirepos tend to be even less efficient (about versioning) than those that use monorepos. What are those far simpler, better, more open alternatives?

Proper secure development lifecycle approach for all software projects producing code?

Re: Monorepoize – Bash scripts for creating a monorepo out of smaller repos

#13

Post by cryptica »

beagle3 wrote: post #9
cryptica wrote: post #7

Big corporations are extremely inefficient. Why does everyone want to copy them? And why do we use all the bulky tools they produce when there are far simpler, better and more open alternatives available?

Because it turns out big corporations that use multirepos tend to be even less efficient (about versioning) than those that use monorepos. What are those far simpler, better, more open alternatives?

I think it's because corporations treat developers like untrustworthy and replaceable cogs in a big machine - IMO, this is what's wrong, not multi-repos. If corporations gave their developers responsibility and ownership over the maintenance of specific repos which are heavily depended upon, then multi-repos would not be a problem. It's a great source of pride for developers to feel responsible for something and to take care of upstream developers.

About simpler alternatives, non-corporate tools and libraries are often much better than corporate-built ones. For example, React was built by Facebook and is inferior to VueJS which was built independently yet React is much more popular.

On top of this, I contend that if Evan You (the creator of VueJS) hadn't worked at Google, very few people would be using VueJS today; but the popularity has more to do with public image than technical merit. VueJS is associated with Google even though it was written independently; something so simple and elegant (especially at the time it came out) could only have been written by an indie dev. Too many cooks do spoil the broth unfortunately.

Re: Monorepoize – Bash scripts for creating a monorepo out of smaller repos

#14

Post by Joe8Bit »

stared wrote: post #5

In the last few years, I saw a lot of people using monorepos, and discouraging using submodules. Frankly, I don't know why. For small projects, sure - it it an overhead. For anything larger, I find it useful to encapsulate things - be it installable packages, or if it is not the case - at least other repositories. In the last months I split one project into 4 repos ( https://github.com/Quantum-Game/ ) and couldn't be…

The benefits of monorepos are (IME) all in people and organisation scale. For example, I've found it orders of magnitude easier to align tooling, quality, release and standards on a monorepo with 100's of 1000's of engineers working on it simultaneously.

The point you've made is a good one though, the idea of monorepos at small scale has come up a lot recently and I'm not sure the arguments for it are as compelling (I don't see many compelling arguments _against_ it either mind you). What I _have_ seen though is the pain large orgs go through going from multi-to-mono repos, I think Twitter did it recently and it was pretty painful.

Re: Monorepoize – Bash scripts for creating a monorepo out of smaller repos

#15

Post by beagle3 »

unixhero wrote: post #12
beagle3 wrote: post #9

Earlier quoted context omitted.

Because it turns out big corporations that use multirepos tend to be even less efficient (about versioning) than those that use monorepos. What are those far simpler, better, more open alternatives?

Proper secure development lifecycle approach for all software projects producing code?

What are those?

There are tens of such approaches, many of them practiced at large corps, and in my experience almost none of them deliver the supposed benefits, with few exceptions (NASA, MISRA) that are cost prohibitive except for very narrow fields.

And what does that have to do with “copying large corps” and/or monorepo vs multirepo?

Re: Monorepoize – Bash scripts for creating a monorepo out of smaller repos

#16

Post by beagle3 »

cryptica wrote: post #13
beagle3 wrote: post #9

Earlier quoted context omitted.

Because it turns out big corporations that use multirepos tend to be even less efficient (about versioning) than those that use monorepos. What are those far simpler, better, more open alternatives?

I think it's because corporations treat developers like untrustworthy and replaceable cogs in a big machine - IMO, this is what's wrong, not multi-repos. If corporations gave their developers responsibility and ownership over the maintenance of specific repos which are heavily depended upon, then multi-repos would not be a problem. It's a great source of pride for developers to feel responsible for something and to t…

I disagree about the nature of the problem.

You can’t give sole ownership of a project people depend on to just one person - a bus factor of 1 is unacceptable in most circumstances. Busses happen, people leave, etc. Some things are also too big for one person.

And when you give it to a team of proud capable developers, you will get ego wars, blame throwing, etc. with probability that increases quickly with team size.

And that’s with capable, proud, responsible, cooperative people.

But not all your people will be like that all the time - some will be burnet our, some have no passion and just try to do the minimum to not get fired. Some are sociopaths who actively try to shift work to others and only pride themselves on work they managed to avoid.

Mom and pop software shops can afford non-replaceable-cogs, Google can’t (and afaik didn’t even in the old days when they were run by engineers and not MBAs).

And this whole discussion of copying corps was in the context of mono Vs multi repo....

(For the record, VueJS and Evan You are awesome)

Re: Monorepoize – Bash scripts for creating a monorepo out of smaller repos

#17

Post by hocuspocus »

beagle3 wrote: post #9
cryptica wrote: post #7

Big corporations are extremely inefficient. Why does everyone want to copy them? And why do we use all the bulky tools they produce when there are far simpler, better and more open alternatives available?

Because it turns out big corporations that use multirepos tend to be even less efficient (about versioning) than those that use monorepos. What are those far simpler, better, more open alternatives?

> Because it turns out big corporations that use multirepos tend to be even less efficient (about versioning) than those that use monorepos.

How do you measure that exactly?

Re: Monorepoize – Bash scripts for creating a monorepo out of smaller repos

#19

Post by stared »

edwintorok wrote: post #6
stared wrote: post #5

In the last few years, I saw a lot of people using monorepos, and discouraging using submodules. Frankly, I don't know why. For small projects, sure - it it an overhead. For anything larger, I find it useful to encapsulate things - be it installable packages, or if it is not the case - at least other repositories. In the last months I split one project into 4 repos ( https://github.com/Quantum-Game/ ) and couldn't be…

Once your code grows and you end up with too many repos you'll be yearning to get back to monorepos. It allows to make changes in one go instead of having 3-4 PRs that all have to be merged at same time or otherwise the build breaks. Having separate repos only makes sense if they are maintained by separate teams with a well defined API in-between that doesn't change often. If the same team maintains all repos then it…

I saw some projects in which my instinct was "too many repos" (e.g. https://github.com/ActivityWatch/activitywatch), exactly for the reasons you mentioned.

However, in my case - as the project gets bigger, I appreciate it more. In the first place, it makes code reusable, and forces to encapsulated functionalities as much as possible. (Yes, having non-encapsulated code would be a nightmare).

To get to the extreme, in the Node ecosystem code is pretty much structured as many, many small packages. (Bear in mind, that turning something into a package is a much stricter requirement that making it effectively a subfolder.)

Re: Monorepoize – Bash scripts for creating a monorepo out of smaller repos

#20

Post by beagle3 »

cryptica wrote: post #8
edwintorok wrote: post #6

Earlier quoted context omitted.

Once your code grows and you end up with too many repos you'll be yearning to get back to monorepos. It allows to make changes in one go instead of having 3-4 PRs that all have to be merged at same time or otherwise the build breaks. Having separate repos only makes sense if they are maintained by separate teams with a well defined API in-between that doesn't change often. If the same team maintains all repos then it…

I maintain such a project with many dependencies which I also maintain separately. Yes it can be a lot of extra work to publish but this argument is based on the assumption that dependencies need to be changed constantly and this assumption is wrong. There should be pressure to design dependencies in such a way that they are modular and don't need to be changed often - This unpleasantness associated with updating dee…

That depends on the scale of the project, of course - if you have thousands of engineers as MicroFaceGoog does, then it’a a daily, even hourly, occurrence for many dependencies.

You are not MicroFaceGood, of course, so you only do that every 6 months.

However, I what way are separate repos nicer for you? Personally, I settled on a monorepo with a “$dep-devel” branch for every dependency, which I occasionally have on multiple working trees out at the same time, and which I feel gives me all the benefit a multirepo could have (whatever they may be, I can’t find them) while still making tracking, branching and merging across dependencies trivial.

Post reply on HN