I've been in a project where some (authoritative) people had a tendency to split things into separate repositories for very small things, e.g. repositories with a single class. This was pure developer hell. Any change meant changing at least 3 repositories, including a review for each change.
Never understood this decision as all parts needed to be on the latest version anyway. Caused lots of dependency and versioning issues too.
Monorepos: Please don’t
71–80 of 402 posts
Re: Monorepos: Please don’t
#72Can anyone here explain to me how a monorepo like Google or Facebook handles security? If I pull the repo - I have the entire contents of Google or Facebook? Is that right? Surely that lacks the normal security measures around what must be highly sensitive information, so there must be more to it than I know of?
(there's an acm paper about Google's repo that dives deeper into this).
First thing, you can't just "pull the whole repo" at Google or fb scale. It doesn't fit on a single hard drive.
This means the enitre repo is normally accessed via networked means. As a result, builds can also be done over the network transparently.
So building and testing is done as a different user. That user can have different privileges than the individual requesting the build.
So there is a way to hide source code so that only the output artifacts (compiled binaries) can be accessed.
But I think the other part of that is that that's normally a tiny minority of code.
The other option is of course to live outside the monorepo, as some projects do.
Re: Monorepos: Please don’t
#73Unless you are pure OSS or pure closed source - you end up with a poly-repo strategy regardless as you split open and closed code, suffering the annoyances of both systems.
No, what you end up with is a system for mirroring open source code into your repo, and a system for mirroring commits that should be open source from your code into external repos. All active work still happens in a monorepo.
Re: Monorepos: Please don’t
#74I'm not familiar with how monorepos work in practice, but it seems obvious to me that it's going to complicate everyday tasks. Ready to commit? Whoops, another team made a bunch of commits to their project, and you need to rebase your project before you can commit. (I'm having flashbacks to Clearcase already.) Need to roll back the last two commits you made? Sure, that takes two seconds--oh, wait, another team made m…
Why would you need to rebase or cherrypick unless you and the other team were touching the same files?
Re: Monorepos: Please don’t
#75My problem with polyrepos is that often organizations end up splitting things too finely, and now I'm unable to make a single commit to introduce a feature because my changes have to live across several repositories. Which makes code review more annoying because you have to tab back and forth to see all the context. It's doubly frustrating when I'm (or my team is) the only people working on those repositories, becaus…
OctoLinker really helps when browsing a polyrepo on Github:
https://github.com/OctoLinker/OctoLinker
You can just click the import [project] name and it will switch to the repo.
Re: Monorepos: Please don’t
#76The truth is that you're not going to get to make this decision. If you're starting greenfield, you're going to start a single repo for your project. If that greenfield is the whole company and everything is part of that project, you get a giant monorepo. If greenfield is a new division that's not part of another project, you're going to create a new repo, and now you're in a polyrepo environment.
Which way it goes is determined by the environment, wherein the engineers do the sensible thing at the time. Then you do the engineering to solve the problems with whatever way you went.
Re: Monorepos: Please don’t
#77Can anyone here explain to me how a monorepo like Google or Facebook handles security? If I pull the repo - I have the entire contents of Google or Facebook? Is that right? Surely that lacks the normal security measures around what must be highly sensitive information, so there must be more to it than I know of?
(Worked at google for 2-3 years, under google3 depot) - AFAIK, only few hundreth files are not visible to employees, and certain folks (contractors) maybe limited there too.
You can actually compile, and run in dev/staging certain things, inspect code, click on function, and see callsites, even "debug" from the browser (debugging, is pretty much, if your binary have stepped through a bookmark, then it'll tell you, and you may print out locals from the scope - like vars, etc. - hard to explain in short).
Then "checking out", to put in p4/svn - is not really like that - but you can find pretty much videos, docs explaining it. It's more like - you create a "workspace", "client" (piper has history from p4, so some terms are similar to perforce), and in this client - you "view" the entire depot, and changes you've made are "overlayed". Then you can submit these in a CL (much like perforce).
There is also git (git5) and hg mode, but I've never used them. I've used the "CitC" one (client in the cloud), and was great as I was able to edit, and later edit, even build,... even deploy all from the browser (though prefer real IDE there - like Eclipse/IntelliJ/CLion).
Re: Monorepos: Please don’t
#78Can anyone here explain to me how a monorepo like Google or Facebook handles security? If I pull the repo - I have the entire contents of Google or Facebook? Is that right? Surely that lacks the normal security measures around what must be highly sensitive information, so there must be more to it than I know of?
You aren’t allowed to clone google’s repo to a laptop.
Re: Monorepos: Please don’t
#79Better title would be "Monorepos don't fit with my particular use case."
I strongly agree. I hate this style of blog post.
Telling people what they should or should not do is generally absurd. Every situation is unique and you can't possibly know another project's requirements or acceptable trade-offs.
A better approach, in my opinion, is "Here's what we did and why". The author clearly has experience in the area. Great! Tell me about your problems. Tell me about your attempted solutions and what did or did not work. Tell me what you wish you had done! I'd love to use knowledge of your situation to inform my own decision making.
But don't be surprised if my circumstances are different and lead me to prefer different trade-offs and choose a different solution. That doesn't make me a zealot or an idiot.
Re: Monorepos: Please don’t
#80I’ve found monorepos to be extremely valuable in an immature, high-churn codebase. Need to change a function signature or interface? Cool, global find & replace. At some point monorepos outgrow their usefulness. The sheer amount of files in something that’s 10K+ LOC ( not that large, I know ) warrants breaking apart the codebase into packages. Still, I almost err on the side of monorepos because of the convenience th…
[deleted]